Nov 01 03:03:49.602558 ip-10-0-134-33 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 01 03:03:49.602572 ip-10-0-134-33 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 01 03:03:49.602580 ip-10-0-134-33 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 01 03:03:49.602955 ip-10-0-134-33 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 01 03:03:59.700318 ip-10-0-134-33 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 01 03:03:59.700332 ip-10-0-134-33 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 728bde978cda4083a5ea0beafdcac73d --
Nov 01 03:05:59.195265 ip-10-0-134-33 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 03:05:59.617961 ip-10-0-134-33 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:05:59.617961 ip-10-0-134-33 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 03:05:59.617961 ip-10-0-134-33 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:05:59.617961 ip-10-0-134-33 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624729    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624826    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624832    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624835    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624839    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624843    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.624831 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624846    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624849    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624852    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624855    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624858    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624861    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624863    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624866    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624868    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624871    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624874    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624876    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624879    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624882    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624884    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624887    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624890    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624892    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624895    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.625128 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624898    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624900    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624903    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624906    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624909    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624912    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624920    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624923    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624927    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624929    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624932    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624934    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624937    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624939    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624942    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624946    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624949    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624952    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624955    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.625570 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624958    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624961    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624964    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624967    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624969    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624972    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624975    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624977    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624980    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.624998    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625000    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625003    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625006    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625010    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625013    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625016    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625018    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625021    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625023    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625027    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.626070 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625029    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625032    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625034    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625037    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625040    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625042    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625046    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625049    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625052    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625055    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625058    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625060    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625063    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625066    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625068    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625071    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625074    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625077    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625080    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625083    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.626545 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625086    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625089    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625092    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625094    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625097    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625100    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625102    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625105    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625108    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625110    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625113    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625117    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625122    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625125    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625128    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625133    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625137    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625140    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625143    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.627047 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625145    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625149    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.625152    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627278    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627287    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627291    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627294    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627297    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627299    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627302    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627304    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627307    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627310    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627313    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627316    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627318    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627321    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627323    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627327    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.627487 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627331    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627334    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627337    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627340    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627343    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627345    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627348    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627351    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627354    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627357    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627359    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627362    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627365    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627368    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627371    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627374    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627376    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627380    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627383    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627385    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.627940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627388    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627391    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627393    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627396    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627398    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627401    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627404    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627407    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627410    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627413    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627417    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627421    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627424    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627427    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627430    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627432    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627435    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627437    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627442    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.628484 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627444    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627447    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627449    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627452    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627455    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627458    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627460    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627463    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627465    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627468    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627471    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627473    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627476    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627478    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627481    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627484    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627487    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627489    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627492    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.628943 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627495    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627498    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627502    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627505    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627508    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627511    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627513    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627516    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627518    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627521    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627523    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627526    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627528    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627531    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627533    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627536    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627538    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627541    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627543    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.629413 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627546    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627549    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627552    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627555    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627558    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627560    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627565    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627567    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627569    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627572    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627574    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627577    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.627580    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628123    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628130    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628134    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628137    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628140    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628143    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628145    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.629863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628148    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628151    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628154    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628156    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628159    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628162    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628165    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628167    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628170    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628172    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628175    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628177    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628180    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628183    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628185    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628189    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628193    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628196    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628199    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628201    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.630348 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628204    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628206    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628209    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628212    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628214    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628217    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628220    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628222    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628227    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628229    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628232    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628234    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628237    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628240    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628242    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628245    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628247    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628249    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628252    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.630822 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628255    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628257    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628260    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628262    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628265    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628268    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628270    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628273    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628275    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628278    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628280    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628283    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628285    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628288    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628290    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628292    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628295    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628297    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628300    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628302    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.631386 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628305    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628308    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628311    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628313    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628316    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628318    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628321    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628323    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628326    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628328    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628331    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628333    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628336    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628338    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628341    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628343    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628346    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628348    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628351    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628353    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.631872 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628356    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628359    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628361    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628364    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628366    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628368    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628371    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628373    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628376    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628378    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628381    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628384    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628386    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628389    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628391    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628394    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628397    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628400    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628402    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.632378 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.628407    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629760    2567 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629777    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629784    2567 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629789    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629793    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629797    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629802    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629807    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629810    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629813    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629817    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629821    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629824    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629827    2567 flags.go:64] FLAG: --cgroup-root=""
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629830    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629834    2567 flags.go:64] FLAG: --client-ca-file=""
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629837    2567 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629840    2567 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629846    2567 flags.go:64] FLAG: --cluster-domain=""
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629849    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629852    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629855    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629858    2567 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 03:05:59.632829 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629863    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629867    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629870    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629873    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629877    2567 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629883    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629886    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629889    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629892    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629896    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629900    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629902    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629905    2567 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629908    2567 flags.go:64] FLAG: --enable-server="true"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629911    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629916    2567 flags.go:64] FLAG: --event-burst="100"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629919    2567 flags.go:64] FLAG: --event-qps="50"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629922    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629925    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629928    2567 flags.go:64] FLAG: --eviction-hard=""
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629933    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629936    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629939    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629942    2567 flags.go:64] FLAG: --eviction-soft=""
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629945    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 03:05:59.633407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629948    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629951    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629953    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629956    2567 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629959    2567 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629962    2567 flags.go:64] FLAG: --feature-gates=""
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629966    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629969    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629973    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629987    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629991    2567 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629994    2567 flags.go:64] FLAG: --help="false"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.629998    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630002    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630006    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630009    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630012    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630015    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630018    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630021    2567 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630024    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630032    2567 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630035    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 03:05:59.634054 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630038    2567 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630041    2567 flags.go:64] FLAG: --kube-reserved=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630044    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630047    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630050    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630053    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630056    2567 flags.go:64] FLAG: --lock-file=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630059    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630062    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630065    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630071    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630074    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630077    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630080    2567 flags.go:64] FLAG: --logging-format="text"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630082    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630086    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630089    2567 flags.go:64] FLAG: --manifest-url=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630092    2567 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630097    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630100    2567 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630104    2567 flags.go:64] FLAG: --max-pods="110"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630109    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630112    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630118    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630122    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 03:05:59.634590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630125    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630128    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630131    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630139    2567 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630143    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630145    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630148    2567 flags.go:64] FLAG: --pod-cidr=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630151    2567 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630154    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630157    2567 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630160    2567 flags.go:64] FLAG: --port="10250"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630163    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630166    2567 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0f20d9680c708fe1c"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630169    2567 flags.go:64] FLAG: --qos-reserved=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630172    2567 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630175    2567 flags.go:64] FLAG: --register-node="true"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630178    2567 flags.go:64] FLAG: --register-with-taints=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630182    2567 flags.go:64] FLAG: --registry-burst="10"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630185    2567 flags.go:64] FLAG: --registry-qps="5"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630188    2567 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630191    2567 flags.go:64] FLAG: --reserved-memory=""
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630195    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630198    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630201    2567 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630204    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630207    2567 flags.go:64] FLAG: --runonce="false"
Nov 01 03:05:59.635207 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630210    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630213    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630217    2567 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630220    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630223    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630230    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630233    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630236    2567 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630239    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630242    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630245    2567 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630250    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630253    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630256    2567 flags.go:64] FLAG: --system-cgroups=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630259    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630265    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630268    2567 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630271    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630275    2567 flags.go:64] FLAG: --tls-min-version=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630279    2567 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630281    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630284    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630287    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630290    2567 flags.go:64] FLAG: --v="2"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630295    2567 flags.go:64] FLAG: --version="false"
Nov 01 03:05:59.635826 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630299    2567 flags.go:64] FLAG: --vmodule=""
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630304    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630307    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630478    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630482    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630485    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630488    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630490    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630494    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630497    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630500    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630502    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630505    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630513    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630516    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630519    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630521    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630524    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630528    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630531    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.636488 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630533    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630536    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630538    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630541    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630543    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630546    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630548    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630551    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630553    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630556    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630558    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630561    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630563    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630566    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630568    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630571    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630574    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630576    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630579    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630582    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.636972 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630584    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630587    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630589    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630592    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630594    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630597    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630605    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630608    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630610    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630613    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630617    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630619    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630622    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630624    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630627    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630629    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630632    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630636    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630639    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.637500 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630642    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630645    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630648    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630650    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630653    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630656    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630658    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630661    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630664    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630666    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630669    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630671    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630674    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630677    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630679    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630682    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630684    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630687    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630689    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630692    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.637940 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630700    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630703    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630706    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630709    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630712    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630715    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630717    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630719    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630722    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630725    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630727    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630729    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630732    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630734    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630737    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630740    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630744    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630747    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630750    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.638427 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630753    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630756    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630758    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630761    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630764    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630766    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630769    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630771    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630774    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630776    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630779    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.630806    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630850    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630855    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630862    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630865    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.638914 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630869    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630872    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630874    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630877    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630880    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630882    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630885    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630888    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630890    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630892    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630895    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630898    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630900    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630903    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630905    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630908    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630910    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630913    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630916    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630918    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.639316 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630921    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630925    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630927    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630930    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630933    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630935    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630938    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630940    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630943    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630945    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630948    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630956    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630960    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630963    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630965    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630968    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630970    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630973    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630987    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.639786 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630989    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630993    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.630998    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631001    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631004    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631006    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631009    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631012    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631015    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631017    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631020    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631022    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631025    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631027    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631032    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631034    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631037    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631039    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631042    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631044    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.640256 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631047    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631050    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631052    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631056    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631058    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631063    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631065    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631068    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631071    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631073    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631076    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631078    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631081    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631084    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631086    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631089    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631092    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631095    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.640724 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631097    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631101    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631104    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631107    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631110    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631112    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631115    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631118    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631122    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631124    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631127    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631129    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631132    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631134    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631137    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631140    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631142    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631145    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631147    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.641170 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631151    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631164    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631167    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631170    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631172    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.631175    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.638326    2567 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.638342    2567 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.638900    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.638906    2567 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.638969    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.638975    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.638994    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.638999    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639002    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639005    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639007    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639011    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639014    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639017    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.641618 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639020    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639022    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639025    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639027    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639031    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639035    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639039    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639042    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639045    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639047    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639050    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639053    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639056    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639058    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639061    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639064    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639067    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639070    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639072    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.642090 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639076    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639079    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639082    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639084    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639087    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639090    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639092    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639095    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639098    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639100    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639103    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639105    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639108    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639111    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639113    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639116    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639119    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639121    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639124    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.642611 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639127    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639129    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639132    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639135    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639137    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639140    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639143    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639145    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639148    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639150    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639154    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639156    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639159    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639163    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639166    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639169    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639171    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639174    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639177    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639179    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.643076 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639182    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639186    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639190    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639193    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639196    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639198    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639201    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639203    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639206    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639208    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639211    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639214    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639216    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639219    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639222    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639224    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639227    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639229    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639232    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.643587 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639235    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639238    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639241    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639243    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639246    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639248    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639251    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639254    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639257    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639260    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639263    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639266    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639268    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639271    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639273    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639276    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639279    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639281    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.644062 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639284    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.639311    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639461    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639468    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639472    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639475    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639478    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639481    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639484    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639486    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639489    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639492    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639494    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639497    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639500    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.644490 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639502    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639505    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639509    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639511    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639514    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639517    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639520    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639522    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639525    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639528    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639531    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639533    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639535    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639538    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639541    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639543    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639546    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639549    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639551    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.644863 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639554    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639556    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639559    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639562    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639564    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639567    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639569    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639571    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639574    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639577    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639579    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639581    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639584    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639586    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639589    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639591    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639594    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639596    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639599    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639602    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.645431 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639604    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639607    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639609    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639612    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639615    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639617    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639620    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639622    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639625    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639627    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639630    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639633    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639635    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639638    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639640    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639642    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639645    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639648    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639650    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639653    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.645912 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639655    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639658    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639660    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639663    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639666    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639668    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639671    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639673    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639677    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639681    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639684    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639687    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639689    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639692    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639694    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639697    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639700    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639703    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639705    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639708    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.646462 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639710    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639712    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639716    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639718    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639721    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639723    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639726    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639728    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639731    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639733    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639736    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639738    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639741    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639743    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.639766    2567 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639798    2567 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:05:59.646938 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639802    2567 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639805    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639808    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639811    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639813    2567 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639816    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639818    2567 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639821    2567 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639823    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639826    2567 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639828    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639831    2567 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639835    2567 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639839    2567 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639843    2567 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639845    2567 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639848    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639851    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639853    2567 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:05:59.647363 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639856    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639858    2567 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639861    2567 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639863    2567 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639866    2567 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639869    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639872    2567 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639875    2567 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639877    2567 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639880    2567 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639882    2567 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639885    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639888    2567 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639891    2567 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639893    2567 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639896    2567 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639898    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639901    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639904    2567 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:05:59.648010 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639906    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639909    2567 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639911    2567 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639914    2567 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639916    2567 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639919    2567 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639921    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639924    2567 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639927    2567 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639929    2567 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639932    2567 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639934    2567 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639937    2567 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639939    2567 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639942    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639944    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639946    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639949    2567 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639952    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639954    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:05:59.648772 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639957    2567 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639959    2567 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639962    2567 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639964    2567 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639967    2567 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639969    2567 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639972    2567 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639974    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639989    2567 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639993    2567 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.639997    2567 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640000    2567 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640002    2567 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640005    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640007    2567 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640010    2567 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640013    2567 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640015    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640024    2567 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640028    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:05:59.649522 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640032    2567 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640035    2567 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640037    2567 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640040    2567 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640042    2567 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640045    2567 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640048    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640052    2567 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640055    2567 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640057    2567 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640060    2567 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640063    2567 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640066    2567 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640070    2567 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640073    2567 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640076    2567 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640079    2567 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640081    2567 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640084    2567 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640087    2567 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:05:59.650017 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640089    2567 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640092    2567 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640095    2567 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640097    2567 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640100    2567 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640102    2567 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:05:59.640105    2567 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.640163    2567 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.642186    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.643141    2567 server.go:1014] "Starting client certificate rotation"
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.643256    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.643283    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 03:05:59.650529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.645518    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 03:05:59.651363 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.651344    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 03:05:59.663407 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.663387    2567 log.go:25] "Validated CRI v1 runtime API"
Nov 01 03:05:59.666992 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.666967    2567 log.go:25] "Validated CRI v1 image API"
Nov 01 03:05:59.667869 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.667849    2567 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 03:05:59.673286 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.673259    2567 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 98f1df92-0359-4698-8cf1-205e8d803fea:/dev/nvme0n1p3 a7fb9b66-8ff1-4406-a7e8-bfcf68ed423e:/dev/nvme0n1p4]
Nov 01 03:05:59.673355 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.673284    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 01 03:05:59.677594 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.677487    2567 manager.go:217] Machine: {Timestamp:2026-11-01 03:05:59.677007784 +0000 UTC m=+0.356738642 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091470 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c706d71ed0668ba079ff590c42da7 SystemUUID:ec2c706d-71ed-0668-ba07-9ff590c42da7 BootID:728bde97-8cda-4083-a5ea-0beafdcac73d Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:5d:63:10:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:5d:63:10:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:42:25:61:13:cd:19 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 03:05:59.677594 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.677592    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 03:05:59.677711 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.677699    2567 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 03:05:59.681645 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.681615    2567 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 03:05:59.681794 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.681647    2567 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-33.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 03:05:59.682241 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.682231    2567 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 03:05:59.682272 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.682244    2567 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 03:05:59.682272 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.682257    2567 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 03:05:59.682272 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.682270    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 03:05:59.683005 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.682994    2567 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 03:05:59.686773 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.686760    2567 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 03:05:59.686891 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.686882    2567 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 03:05:59.687671 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.687661    2567 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 03:05:59.687710 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.687675    2567 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 03:05:59.687710 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.687686    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 03:05:59.687710 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.687696    2567 kubelet.go:405] "Adding apiserver pod source"
Nov 01 03:05:59.687710 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.687706    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 03:05:59.688097 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.688082    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 03:05:59.690513 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.690497    2567 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 03:05:59.694648 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.694635    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 03:05:59.695409 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695397    2567 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 03:05:59.695460 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695428    2567 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 03:05:59.695605 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695591    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 03:05:59.695637 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695622    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 03:05:59.695669 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695636    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 03:05:59.695669 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695649    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 03:05:59.695669 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695665    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 03:05:59.695748 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.695680    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 03:05:59.696236 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696227    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 03:05:59.696273 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696236    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 03:05:59.696273 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696244    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 03:05:59.696273 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696250    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 03:05:59.696273 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696259    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 03:05:59.696273 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.696268    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 03:05:59.697949 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.697938    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 03:05:59.698004 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.697952    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 03:05:59.704845 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.704832    2567 server.go:1263] "Started kubelet"
Nov 01 03:05:59.705137 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.704951    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 03:05:59.708438 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.705106    2567 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 03:05:59.708694 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.708651    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 03:05:59.709417 ip-10-0-134-33 systemd[1]: Started Kubernetes Kubelet.
Nov 01 03:05:59.712135 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.712117    2567 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 03:05:59.712729 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.712715    2567 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 03:05:59.714186 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.714169    2567 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 01 03:05:59.716787 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.716767    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 03:05:59.717278 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.717262    2567 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 03:05:59.717366 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.717346    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:05:59.718055 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.717868    2567 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 03:05:59.718055 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.717886    2567 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 03:05:59.718055 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.718032    2567 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 03:05:59.718055 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.718040    2567 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 03:05:59.719026 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.719005    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 03:05:59.720109 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720093    2567 factory.go:55] Registering systemd factory
Nov 01 03:05:59.720109 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720111    2567 factory.go:223] Registration of the systemd container factory successfully
Nov 01 03:05:59.720345 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720332    2567 factory.go:153] Registering CRI-O factory
Nov 01 03:05:59.720378 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720347    2567 factory.go:223] Registration of the crio container factory successfully
Nov 01 03:05:59.720409 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720389    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 03:05:59.720409 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720407    2567 factory.go:103] Registering Raw factory
Nov 01 03:05:59.720478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.720427    2567 manager.go:1199] Started watching for new ooms in manager
Nov 01 03:05:59.721298 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.721285    2567 manager.go:319] Starting recovery of all containers
Nov 01 03:05:59.730945 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.730928    2567 manager.go:324] Recovery completed
Nov 01 03:05:59.735590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.735469    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.741814 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.741797    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.741897 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.741831    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.741897 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.741844    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.742350 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.742332    2567 cpu_manager.go:226] "Starting" policy="none"
Nov 01 03:05:59.742350 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.742347    2567 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 03:05:59.742350 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.742350    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.742484 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.742363    2567 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 03:05:59.743109 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.743094    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.743172 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.743119    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.743172 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.743129    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.744649 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.744637    2567 policy_none.go:50] "Start"
Nov 01 03:05:59.744703 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.744652    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 03:05:59.744703 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.744662    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 03:05:59.746732 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.746714    2567 policy_none.go:44] "Start"
Nov 01 03:05:59.789736 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.789688    2567 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 03:05:59.793989 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.793961    2567 manager.go:341] "Starting Device Plugin manager"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.794014    2567 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794026    2567 server.go:87] "Starting device plugin registration server"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794358    2567 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794370    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794463    2567 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794584    2567 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.794593    2567 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.796831    2567 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 03:05:59.814016 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.796874    2567 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:05:59.841332 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.841293    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 03:05:59.842487 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.842469    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 03:05:59.842567 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.842492    2567 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 03:05:59.842567 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.842512    2567 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 03:05:59.842567 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.842546    2567 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Nov 01 03:05:59.849360 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.849316    2567 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-33.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Nov 01 03:05:59.849360 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.849328    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-33.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 01 03:05:59.850300 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.849326    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-33.us-west-1.compute.internal.18e3cce4ce07765d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-33.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-33.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 03:05:59.704794717 +0000 UTC m=+0.384525568,LastTimestamp:2026-11-01 03:05:59.704794717 +0000 UTC m=+0.384525568,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-33.us-west-1.compute.internal,}"
Nov 01 03:05:59.895125 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.894923    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.895967 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.895947    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.896088 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.896001    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.896088 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.896026    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.896088 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.896059    2567 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:05:59.905214 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.905192    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-33.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 01 03:05:59.905309 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.905149    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-33.us-west-1.compute.internal.18e3cce4d03c56bc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-33.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-33.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-33.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 03:05:59.74181446 +0000 UTC m=+0.421545310,LastTimestamp:2026-11-01 03:05:59.74181446 +0000 UTC m=+0.421545310,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-33.us-west-1.compute.internal,}"
Nov 01 03:05:59.905309 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.905282    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 01 03:05:59.905402 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.905315    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Nov 01 03:05:59.943295 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.943268    2567 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"]
Nov 01 03:05:59.943393 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.943333    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.945201 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.945182    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.945305 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.945218    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.945305 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.945231    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.947211 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.947198    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.947374 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.947357    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:05:59.947441 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.947399    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.947994 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.947955    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.947994 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.947992    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.948122 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.948006    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.948122 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.948050    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.948122 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.948071    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.948122 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.948084    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.950038 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.950023    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:05:59.950108 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.950054    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:05:59.951053 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.951035    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:05:59.951112 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.951068    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:05:59.951112 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.951083    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:05:59.959225 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.959207    2567 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 03:05:59.959313 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:05:59.959258    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zgrzs"
Nov 01 03:05:59.981303 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.981281    2567 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:05:59.984193 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:05:59.984176    2567 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.007065 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.007038    2567 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass"
Nov 01 03:06:00.007200 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.007091    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-33.us-west-1.compute.internal.18e3cce4d03cb181  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-33.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-33.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-33.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 03:05:59.741837697 +0000 UTC m=+0.421568547,LastTimestamp:2026-11-01 03:05:59.741837697 +0000 UTC m=+0.421568547,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-33.us-west-1.compute.internal,}"
Nov 01 03:06:00.071428 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.071394    2567 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.120298 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.120263    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.120298 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.120298    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.120488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.120317    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.155802 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.155747    2567 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220735 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220704    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220803 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220738    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220803 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220763    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220894 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220799    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220894 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220828    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.220894 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.220803    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.271957 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.271914    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:06:00.272967 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.272948    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:06:00.273075 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.272992    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:06:00.273075 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.273007    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:06:00.273075 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.273044    2567 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.284134 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.284115    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.288954 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.288624    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.329750 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.329722    2567 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:00.329750 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.329748    2567 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-33.us-west-1.compute.internal\": node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.460411 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.460167    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.560711 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.560676    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.643400 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.643365    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 01 03:06:00.643893 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.643543    2567 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a28e94d7245cf43b9a1c835832cbb3a3-03ecbaf5388fcb74.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-33.us-west-1.compute.internal.18e3cce4d03cd988\": read tcp 10.0.134.33:40182->18.225.44.192:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-33.us-west-1.compute.internal.18e3cce4d03cd988  default   6033 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-33.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-33.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-33.us-west-1.compute.internal,},FirstTimestamp:2026-11-01 03:05:59 +0000 UTC,LastTimestamp:2026-11-01 03:05:59.896032068 +0000 UTC m=+0.575762921,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-33.us-west-1.compute.internal,}"
Nov 01 03:06:00.661974 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.661946    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.763119 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.763031    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.863554 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.863522    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.964005 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:00.963966    2567 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:06:00.964113 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.963997    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 03:01:00 +0000 UTC" deadline="2028-07-13 05:08:18.750668696 +0000 UTC"
Nov 01 03:06:00.964113 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:00.964022    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14882h2m17.78665002s"
Nov 01 03:06:01.019266 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.019209    2567 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:06:01.020268 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.020251    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:06:01.020347 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.020282    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:06:01.020347 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.020295    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:06:01.028805 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.028789    2567 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:06:01.118508 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.118470    2567 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:01.185666 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.185641    2567 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:06:01.187534 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.187519    2567 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:06:01.252741 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.252712    2567 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:06:01.467583 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:01.467542    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf94e9eb892b28831be741642f228e6d.slice/crio-2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314 WatchSource:0}: Error finding container 2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314: Status 404 returned error can't find the container with id 2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314
Nov 01 03:06:01.475882 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.475864    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:06:01.476989 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:01.476966    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod726f6b8e587032dd5c6b3f5ea9e72ff8.slice/crio-b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d WatchSource:0}: Error finding container b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d: Status 404 returned error can't find the container with id b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d
Nov 01 03:06:01.593064 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.593030    2567 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:06:01.593263 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.593186    2567 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:06:01.647729 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.647650    2567 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:06:01.689602 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.689575    2567 apiserver.go:52] "Watching apiserver"
Nov 01 03:06:01.720089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.720041    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 01 03:06:01.803330 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.803304    2567 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 03:06:01.803709 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.803686    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-nrvct","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f","openshift-cluster-node-tuning-operator/tuned-pnjx6","openshift-dns/node-resolver-8ddz9","openshift-multus/network-metrics-daemon-9r9sn","openshift-network-diagnostics/network-check-target-h2vwz","kube-system/global-pull-secret-syncer-kkdq4","kube-system/konnectivity-agent-d72l5","kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal","openshift-network-operator/iptables-alerter-frqbw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal","openshift-multus/multus-additional-cni-plugins-pw9lf","openshift-image-registry/node-ca-2lbxr","openshift-multus/multus-n6qp4"]
Nov 01 03:06:01.807595 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.807569    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.809916 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.809896    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.812357 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.812334    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.814606 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.814585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.816972 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.816951    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:01.817081 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:01.817055    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:01.820029 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.820009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:01.820139 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:01.820090    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:01.822254 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.822236    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:01.822331 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:01.822293    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:01.824588 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.824572    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:01.829261 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.829237    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:01.830029 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830005    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.830190 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830072    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830190 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830313 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830222    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830313 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830408 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttw5v\" (UniqueName: \"kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.830408 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830341    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.830472 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.830510 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.830561 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830539    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:01.830660 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830660 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.830749 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.830731    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831150 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831119    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqpfb\" (UniqueName: \"kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831264 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831163    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.831264 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831190    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831264 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831219    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831282    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831349    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831440    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831463    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsrhz\" (UniqueName: \"kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.831496 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831489    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rlhw\" (UniqueName: \"kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831523    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831570    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831645    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831717    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831751    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfwbf\" (UniqueName: \"kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831787    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831831    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.831873 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831878    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831910    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.831956    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832006    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832045    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832090    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832162    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.832430 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.832235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.833647 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.833630    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:01.835986 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.835968    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.846723 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.846676    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"726f6b8e587032dd5c6b3f5ea9e72ff8","Type":"ContainerStarted","Data":"b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d"}
Nov 01 03:06:01.847920 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.847899    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerStarted","Data":"2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314"}
Nov 01 03:06:01.873539 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873515    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 03:06:01.873539 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873523    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873588    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873613    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873617    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873723    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873756    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873771    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4hc7l\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873796    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ksxvj\""
Nov 01 03:06:01.873887 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873835    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873897    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-77n5m\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.873901    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mt5gq\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.874135    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.874146    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.874141    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.874177    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.874321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.874192    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.920920 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.920828    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 03:06:01.925134 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.925111    2567 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 03:06:01.925252 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.925183    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-7bsjq"
Nov 01 03:06:01.932478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932449    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932477    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932493    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932516    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932538    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932572    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932572    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932636 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932592    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932649    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932696    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttw5v\" (UniqueName: \"kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932773    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932819    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932803    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932871    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmqwm\" (UniqueName: \"kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932945    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.932951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932951    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.932995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933069    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933076    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933119    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933131    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933148    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc5h9\" (UniqueName: \"kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933200    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933178    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rlhw\" (UniqueName: \"kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.933563 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933324    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933272    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933695    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933711    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933727    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933741    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933760    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933779    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933797    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933805    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933814    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwbf\" (UniqueName: \"kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933839    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933851    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933875    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.933994    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934011    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934290 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934018    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934022    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934085    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934101    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934157    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934180    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934184    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934162    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934255    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934299    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934316    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934339    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934348    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934364    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.934984 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934405    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqpfb\" (UniqueName: \"kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934473    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934480    2567 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934486    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrhz\" (UniqueName: \"kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:01.934520    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:01.934673    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:02.434651566 +0000 UTC m=+3.114382425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.934549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935067    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935086    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935175    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:01.935819 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935208    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935218    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935262    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935269    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935279    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935301    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935328    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935337    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935357    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935388    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxkth\" (UniqueName: \"kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935413    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935455    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935468    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935490    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbrgx\" (UniqueName: \"kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:01.936644 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935547    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935571    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935591    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935615    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935646    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935654    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935658    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935771    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935836    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935861    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.935978    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.936043    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.936091    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.937548 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.936150    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.938348 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.938310    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.938348 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.938332    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:01.938523 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.938504    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:01.980460 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980405    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980476    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980510    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980533    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980566    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980590    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kbjpv\""
Nov 01 03:06:01.980692 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980610    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980780    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980838    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980855    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6592m\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980867    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980927    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980933    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980947    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980970    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.980974    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62njk\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.981024    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hrmfk\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.981051    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vzcml\""
Nov 01 03:06:01.981089 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:01.981057    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 03:06:02.000703 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.000676    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:02.000703 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.000704    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:02.000857 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.000718    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:02.000857 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.000806    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:02.500789383 +0000 UTC m=+3.180520258 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:02.002913 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.002888    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwbf\" (UniqueName: \"kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:02.003011 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.002989    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrhz\" (UniqueName: \"kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:02.008747 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.008726    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rlhw\" (UniqueName: \"kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:02.008846 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.008833    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqpfb\" (UniqueName: \"kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:02.008920 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.008894    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttw5v\" (UniqueName: \"kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:02.036406 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036366    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036459    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036529 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036516    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkth\" (UniqueName: \"kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036702 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrgx\" (UniqueName: \"kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.036702 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036606    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.036702 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036702 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036473    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036702 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036666    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036702    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036717    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036742    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036771    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036790    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036815    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036838    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036874    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036900    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036908    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.036946 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036846    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036954    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.036973    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqwm\" (UniqueName: \"kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037050    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037080    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5h9\" (UniqueName: \"kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037128    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037155    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037181    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037246    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037279    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037279    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037305    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037326    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037325    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.037478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037249    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037352    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037365    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037369    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037424    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.037469    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037483    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.037530    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:02.537510888 +0000 UTC m=+3.217241756 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037552    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037587    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037645    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037771    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037883    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.037961    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038081    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038147    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038191    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038303    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038864    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038874    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.038951 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.038874    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.039151 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.039008    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.039186 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.039157    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:02.039348 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.039330    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:02.122360 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.122312    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrgx\" (UniqueName: \"kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.124118 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.124093    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkth\" (UniqueName: \"kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.124347 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.124324    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5h9\" (UniqueName: \"kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.124689 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.124672    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqwm\" (UniqueName: \"kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.139846 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.139823    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:06:02.142160 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.142140    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:06:02.145488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.145473    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:02.146164 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.146147    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:06:02.151588 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.151561    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26425cb6_d11f_4434_ba47_732c8fe23280.slice/crio-4c85ccc3ed049221c7efd4607459bafb1e3671448dc7eeff8ef99d319bdc74fc WatchSource:0}: Error finding container 4c85ccc3ed049221c7efd4607459bafb1e3671448dc7eeff8ef99d319bdc74fc: Status 404 returned error can't find the container with id 4c85ccc3ed049221c7efd4607459bafb1e3671448dc7eeff8ef99d319bdc74fc
Nov 01 03:06:02.156760 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.156739    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00459c43_88b4_494f_a76f_3cbef932a916.slice/crio-d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5 WatchSource:0}: Error finding container d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5: Status 404 returned error can't find the container with id d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5
Nov 01 03:06:02.158328 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.158210    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69348615_8e95_479b_b360_6991eff92a2e.slice/crio-be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d WatchSource:0}: Error finding container be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d: Status 404 returned error can't find the container with id be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d
Nov 01 03:06:02.159197 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.159125    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:02.160606 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.160585    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb94a967_7b3e_45b4_b117_0b591882dc95.slice/crio-93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6 WatchSource:0}: Error finding container 93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6: Status 404 returned error can't find the container with id 93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6
Nov 01 03:06:02.161461 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.161400    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:06:02.166683 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.166661    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:06:02.168850 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.168831    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:06:02.170333 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.170308    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43635a20_c47b_45c7_9616_8503d796561e.slice/crio-9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00 WatchSource:0}: Error finding container 9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00: Status 404 returned error can't find the container with id 9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00
Nov 01 03:06:02.171026 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.170976    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-n6qp4"
Nov 01 03:06:02.173515 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.173489    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc70ed0_1987_4933_9c27_e98b5b15dc32.slice/crio-540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825 WatchSource:0}: Error finding container 540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825: Status 404 returned error can't find the container with id 540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825
Nov 01 03:06:02.178659 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.178637    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18679b6e_26b2_4b4d_8169_d30c6e45d201.slice/crio-01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037 WatchSource:0}: Error finding container 01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037: Status 404 returned error can't find the container with id 01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037
Nov 01 03:06:02.180446 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.180421    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4e4f55_5014_48d0_98cc_669a1cce1dd8.slice/crio-04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8 WatchSource:0}: Error finding container 04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8: Status 404 returned error can't find the container with id 04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8
Nov 01 03:06:02.183433 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:06:02.183388    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b2e9266_2c96_49c3_8d2f_763aeb410cd2.slice/crio-fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3 WatchSource:0}: Error finding container fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3: Status 404 returned error can't find the container with id fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3
Nov 01 03:06:02.441361 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.441278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:02.441523 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.441437    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:02.441523 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.441504    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:03.441487283 +0000 UTC m=+4.121218145 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:02.541788 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.541748    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:02.541953 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.541826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:02.541953 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.541879    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:02.542063 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.541960    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:03.54194557 +0000 UTC m=+4.221676411 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:02.542063 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.541967    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:02.542063 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.541980    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:02.542063 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.541991    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:02.542063 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:02.542056    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:03.542041039 +0000 UTC m=+4.221771895 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:02.851294 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.851184    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f" event={"ID":"00459c43-88b4-494f-a76f-3cbef932a916","Type":"ContainerStarted","Data":"d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5"}
Nov 01 03:06:02.852851 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.852821    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerStarted","Data":"01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037"}
Nov 01 03:06:02.854253 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.854227    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frqbw" event={"ID":"2bc70ed0-1987-4933-9c27-e98b5b15dc32","Type":"ContainerStarted","Data":"540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825"}
Nov 01 03:06:02.860543 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.860489    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d72l5" event={"ID":"43635a20-c47b-45c7-9616-8503d796561e","Type":"ContainerStarted","Data":"9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00"}
Nov 01 03:06:02.862360 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.862331    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8ddz9" event={"ID":"cb94a967-7b3e-45b4-b117-0b591882dc95","Type":"ContainerStarted","Data":"93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6"}
Nov 01 03:06:02.864061 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.864036    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d"}
Nov 01 03:06:02.865382 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.865351    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pnjx6" event={"ID":"26425cb6-d11f-4434-ba47-732c8fe23280","Type":"ContainerStarted","Data":"4c85ccc3ed049221c7efd4607459bafb1e3671448dc7eeff8ef99d319bdc74fc"}
Nov 01 03:06:02.867035 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.867012    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n6qp4" event={"ID":"8b2e9266-2c96-49c3-8d2f-763aeb410cd2","Type":"ContainerStarted","Data":"fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3"}
Nov 01 03:06:02.868301 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.868277    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lbxr" event={"ID":"1a4e4f55-5014-48d0-98cc-669a1cce1dd8","Type":"ContainerStarted","Data":"04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8"}
Nov 01 03:06:02.928134 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.928076    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 03:01:02 +0000 UTC" deadline="2028-04-25 10:40:38.319520363 +0000 UTC"
Nov 01 03:06:02.928134 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:02.928125    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12991h34m35.391397497s"
Nov 01 03:06:03.449152 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.449114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:03.449375 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.449305    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:03.449375 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.449371    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:05.449349445 +0000 UTC m=+6.129080304 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:03.550212 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.550174    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:03.550404 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.550259    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:03.550482 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550423    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:03.550482 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550442    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:03.550482 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550454    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:03.550641 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550497    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:03.550641 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550508    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:05.550490392 +0000 UTC m=+6.230221239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:03.550641 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.550588    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:05.550561259 +0000 UTC m=+6.230292107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:03.845478 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.845405    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:03.845656 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.845545    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:03.845731 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.845664    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:03.845853 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.845828    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:03.846016 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.845934    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:03.846089 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:03.846016    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:03.928366 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.928317    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 03:01:02 +0000 UTC" deadline="2028-05-27 19:30:07.599551738 +0000 UTC"
Nov 01 03:06:03.928366 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:03.928362    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13768h24m3.671194965s"
Nov 01 03:06:05.471608 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.471572    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:05.472170 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.471768    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:05.472170 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.471838    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:09.471818648 +0000 UTC m=+10.151549502 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.572099    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.572199    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572361    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572381    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572395    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572457    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:09.572438382 +0000 UTC m=+10.252169239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572878    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:05.572968 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.572931    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:09.57291238 +0000 UTC m=+10.252643243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:05.845690 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.845660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:05.845850 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.845827    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:05.845931 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.845901    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:05.845989 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.845970    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:05.846060 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:05.846005    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:05.846113 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:05.846049    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:06.891650 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:06.890677    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"726f6b8e587032dd5c6b3f5ea9e72ff8","Type":"ContainerStarted","Data":"a111cd2dd6e26077d014d13a202f85adc4e7b24ca0369a56ae544924b7382a95"}
Nov 01 03:06:06.896601 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:06.895801    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="d63c61aa03251981a830f4517eb977d3f7ab8786ce28c1053a2e71ba37a3f3d0" exitCode=0
Nov 01 03:06:06.896601 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:06.895898    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"d63c61aa03251981a830f4517eb977d3f7ab8786ce28c1053a2e71ba37a3f3d0"}
Nov 01 03:06:06.913444 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:06.913406    2567 generic.go:356] "Generic (PLEG): container finished" podID="df94e9eb892b28831be741642f228e6d" containerID="e1f6ae699c72c9ed199ddfd7d9b4d7f3f31b536f115d71b7575cee1d248b66ec" exitCode=0
Nov 01 03:06:06.913599 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:06.913504    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerDied","Data":"e1f6ae699c72c9ed199ddfd7d9b4d7f3f31b536f115d71b7575cee1d248b66ec"}
Nov 01 03:06:07.163033 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.161693    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal" podStartSLOduration=6.161675063 podStartE2EDuration="6.161675063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:07.018122714 +0000 UTC m=+7.697853580" watchObservedRunningTime="2026-11-01 03:06:07.161675063 +0000 UTC m=+7.841405929"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.845157    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:07.845286    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.845458    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:07.845570    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.845647    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:07.845800 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:07.845756    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:07.920930 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.920177    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-d72l5" event={"ID":"43635a20-c47b-45c7-9616-8503d796561e","Type":"ContainerStarted","Data":"1be5885f93f1f266ecc5ba767cb60e5d9fa8c0e509cf4d5896ca73915c8e46f8"}
Nov 01 03:06:07.926844 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:07.926717    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerStarted","Data":"5a08f318f4bdaca2b5ea74bcdad5f7d45562ae08e7e44185ba7235273d5baff7"}
Nov 01 03:06:08.185139 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:08.184898    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-d72l5" podStartSLOduration=2.383467085 podStartE2EDuration="7.184876105s" totalImagesPullingTime="4.80140902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:08.056483793 +0000 UTC m=+8.736214657" watchObservedRunningTime="2026-11-01 03:06:08.184876105 +0000 UTC m=+8.864606972"
Nov 01 03:06:08.931700 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:08.931609    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-2lbxr" event={"ID":"1a4e4f55-5014-48d0-98cc-669a1cce1dd8","Type":"ContainerStarted","Data":"6efa5705c0525a33966e7ad1afc2170528f272fc44bcc09954e35be471b3ebba"}
Nov 01 03:06:08.936247 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:08.936144    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f" event={"ID":"00459c43-88b4-494f-a76f-3cbef932a916","Type":"ContainerStarted","Data":"eddab3ffab25c49ea308b5eaa9599e4e5c79c7087eecd00d5a70adaca9530731"}
Nov 01 03:06:09.060698 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.060636    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" podStartSLOduration=8.060618415 podStartE2EDuration="8.060618415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:08.184391987 +0000 UTC m=+8.864122854" watchObservedRunningTime="2026-11-01 03:06:09.060618415 +0000 UTC m=+9.740349278"
Nov 01 03:06:09.060904 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.060826    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-2lbxr" podStartSLOduration=2.255482436 podStartE2EDuration="8.060818713s" totalImagesPullingTime="5.805336277s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:09.060406266 +0000 UTC m=+9.740137130" watchObservedRunningTime="2026-11-01 03:06:09.060818713 +0000 UTC m=+9.740549594"
Nov 01 03:06:09.509588 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.508949    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:09.509588 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.509174    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:09.509588 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.509248    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:17.509226887 +0000 UTC m=+18.188957734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.609760    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.609845    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.610022    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.610068    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.610082    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:09.610596 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.610146    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:17.610127919 +0000 UTC m=+18.289858759 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:09.611274 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.611131    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:09.611274 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.611239    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:17.611220638 +0000 UTC m=+18.290951494 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:09.762287 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.762143    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:09.847152 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.847121    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:09.847324 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.847167    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:09.847324 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:09.847258    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:09.847430 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.847354    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:09.847587 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.847514    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:09.847587 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:09.847555    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:10.802362 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.802332    2567 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 03:06:10.808230 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.808135    2567 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T03:06:10.802358863Z","UUID":"878a4731-94c7-42ed-b3de-95b44c7203f1","Handler":null,"Name":"","Endpoint":""}
Nov 01 03:06:10.809669 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.809636    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 03:06:10.809669 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.809664    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 03:06:10.942228 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.942173    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-n6qp4" event={"ID":"8b2e9266-2c96-49c3-8d2f-763aeb410cd2","Type":"ContainerStarted","Data":"5920f80766e7edbb90a3972248cc2c967d3282ae5fefe84ba3985d3bca51a565"}
Nov 01 03:06:10.944121 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:10.944091    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f" event={"ID":"00459c43-88b4-494f-a76f-3cbef932a916","Type":"ContainerStarted","Data":"f20a043ce6ad790d10f1924df32b26f8fb2bd9b6fb8a149979d4490bd492e250"}
Nov 01 03:06:11.062821 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.062727    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-n6qp4" podStartSLOduration=1.60060764 podStartE2EDuration="10.062710098s" totalImagesPullingTime="8.462102458s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:11.062574447 +0000 UTC m=+11.742305308" watchObservedRunningTime="2026-11-01 03:06:11.062710098 +0000 UTC m=+11.742440961"
Nov 01 03:06:11.843044 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.843008    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:11.843513 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.843007    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:11.843513 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.843162    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:11.843513 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:11.843155    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:11.843513 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:11.843269    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:11.843513 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:11.843343    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:11.949530 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.949497    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="79b96a001e3c9cee00c61a097fef9eccf2bf38fa8c67e2e205768c7de5fcb5e1" exitCode=0
Nov 01 03:06:11.949698 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.949578    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"79b96a001e3c9cee00c61a097fef9eccf2bf38fa8c67e2e205768c7de5fcb5e1"}
Nov 01 03:06:11.952180 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.951749    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frqbw" event={"ID":"2bc70ed0-1987-4933-9c27-e98b5b15dc32","Type":"ContainerStarted","Data":"1b412d8e2ec63bae9ab08c848b342bacaa2ecd7a78661027fa3b190bf1f078c3"}
Nov 01 03:06:11.954646 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:11.954621    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8ddz9" event={"ID":"cb94a967-7b3e-45b4-b117-0b591882dc95","Type":"ContainerStarted","Data":"261dce3db33d9313eb627aef3116c434fd9aa52751518230e2027eaaa187f44e"}
Nov 01 03:06:12.194491 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.194148    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-8ddz9" podStartSLOduration=2.255006834 podStartE2EDuration="11.194134792s" totalImagesPullingTime="8.939127958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:12.193829203 +0000 UTC m=+12.873560065" watchObservedRunningTime="2026-11-01 03:06:12.194134792 +0000 UTC m=+12.873865654"
Nov 01 03:06:12.310303 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.310236    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-frqbw" podStartSLOduration=3.033512923 podStartE2EDuration="11.310219686s" totalImagesPullingTime="8.276706763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:12.309938959 +0000 UTC m=+12.989669818" watchObservedRunningTime="2026-11-01 03:06:12.310219686 +0000 UTC m=+12.989950547"
Nov 01 03:06:12.653467 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.653430    2567 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:12.654226 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.654206    2567 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:12.959375 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.959295    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f" event={"ID":"00459c43-88b4-494f-a76f-3cbef932a916","Type":"ContainerStarted","Data":"8658324d2e7409f9dfcdd75eb70fab4e781a0a2ba4f435f316ec9948afbb1fba"}
Nov 01 03:06:12.959882 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:12.959869    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:06:13.079567 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.079495    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f" podStartSLOduration=2.164759628 podStartE2EDuration="12.079476327s" totalImagesPullingTime="9.914716699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:13.078934409 +0000 UTC m=+13.758665269" watchObservedRunningTime="2026-11-01 03:06:13.079476327 +0000 UTC m=+13.759207188"
Nov 01 03:06:13.843208 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.843130    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:13.843372 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.843133    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:13.843372 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.843252    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:13.843372 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:13.843257    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:13.843521 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:13.843371    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:13.843521 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:13.843441    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:13.963088 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.963049    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="e27fcad73c876e4c83404a0240b62699a96bf943113574af59a5a96d2f2058b9" exitCode=0
Nov 01 03:06:13.963587 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:13.963135    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"e27fcad73c876e4c83404a0240b62699a96bf943113574af59a5a96d2f2058b9"}
Nov 01 03:06:14.966276 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:14.966239    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-pnjx6" event={"ID":"26425cb6-d11f-4434-ba47-732c8fe23280","Type":"ContainerStarted","Data":"0cc8f904f357cf7618e94aecb98fa266d2e652e124b20a028e93272308016117"}
Nov 01 03:06:15.097302 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.097226    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-pnjx6" podStartSLOduration=2.177571398 podStartE2EDuration="14.097206391s" totalImagesPullingTime="11.919634993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:15.096851515 +0000 UTC m=+15.776582402" watchObservedRunningTime="2026-11-01 03:06:15.097206391 +0000 UTC m=+15.776937252"
Nov 01 03:06:15.843237 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.843043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:15.843428 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.843051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:15.843428 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:15.843328    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:15.843428 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.843081    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:15.843428 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:15.843396    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:15.843568 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:15.843464    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:15.970786 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.970754    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="9f6e6e35127d5b6ac71c392a61697f724d40722770c1b2ba137f879f89f15479" exitCode=0
Nov 01 03:06:15.971194 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:15.970833    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"9f6e6e35127d5b6ac71c392a61697f724d40722770c1b2ba137f879f89f15479"}
Nov 01 03:06:17.569021 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.568965    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:17.569518 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.569128    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:17.569518 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.569212    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:33.569192794 +0000 UTC m=+34.248923651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:17.669830 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.669787    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:17.670023 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.669862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:17.670106 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670017    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:17.670106 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670041    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:17.670106 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670058    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:17.670106 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670071    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:17.670106 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670097    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:33.670079762 +0000 UTC m=+34.349810614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:17.670366 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.670114    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:06:33.670105585 +0000 UTC m=+34.349836422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:17.843749 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.843662    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:17.843749 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.843680    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:17.843749 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:17.843699    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:17.844013 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.843791    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:17.849241 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.849197    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:17.849372 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:17.849346    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:18.980079 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:18.980046    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="a3e0e72e5c3414e8f5b7980c2d180bf95dd90659723ae6d12b151acbf4086810" exitCode=0
Nov 01 03:06:18.980479 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:18.980087    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"a3e0e72e5c3414e8f5b7980c2d180bf95dd90659723ae6d12b151acbf4086810"}
Nov 01 03:06:19.844111 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:19.844080    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:19.844252 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:19.844162    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:19.844252 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:19.844239    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:19.844360 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:19.844342    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:19.844397 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:19.844381    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:19.844433 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:19.844426    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:19.986330 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:19.986295    2567 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="86610775e7c6cf24fa4706795e105a4927b80136dc5570a209568af5040dcf34" exitCode=0
Nov 01 03:06:19.986779 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:19.986351    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerDied","Data":"86610775e7c6cf24fa4706795e105a4927b80136dc5570a209568af5040dcf34"}
Nov 01 03:06:20.995461 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:20.995426    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" event={"ID":"18679b6e-26b2-4b4d-8169-d30c6e45d201","Type":"ContainerStarted","Data":"4227726cc16c062ee2b539ef91bf993700639a4896c7372a86ff129e9bce9c62"}
Nov 01 03:06:21.171797 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:21.171741    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pw9lf" podStartSLOduration=7.272078583 podStartE2EDuration="20.171728064s" totalImagesPullingTime="12.899649481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:21.171188855 +0000 UTC m=+21.850919714" watchObservedRunningTime="2026-11-01 03:06:21.171728064 +0000 UTC m=+21.851458924"
Nov 01 03:06:21.843619 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:21.843583    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:21.843619 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:21.843600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:21.843867 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:21.843730    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:21.843867 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:21.843755    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:21.843962 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:21.843868    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:21.844030 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:21.843946    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:23.843038 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:23.842851    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:23.843446 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:23.842884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:23.843446 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:23.843128    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:23.843446 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:23.843192    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:23.843446 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:23.842923    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:23.843446 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:23.843270    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:24.004998 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.004903    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:06:24.005265 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005237    2567 generic.go:356] "Generic (PLEG): container finished" podID="69348615-8e95-479b-b360-6991eff92a2e" containerID="f46eb3a4b5b71bf310e0a8e7ca53143a3ae34eea271aad8a3e73b679e5cd8d6d" exitCode=1
Nov 01 03:06:24.005327 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005284    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"c466823d2ab6cc02e9e123667547234e5bf88bef274a5e92a66012fd42f33af9"}
Nov 01 03:06:24.005388 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005345    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"f9be3cee28394529d4c7132ee3ee053d938bc422da52ad1462b8fd4e74202935"}
Nov 01 03:06:24.005438 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005388    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"70daae212777bf3c8a438f1573f80fe5a6acb0bde520d8e602fd26c6f6635d3d"}
Nov 01 03:06:24.005438 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005425    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"83e556be3305caefa25720becdb4b197dc4e797ac8e0d6280120ae12bba0767e"}
Nov 01 03:06:24.005536 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005452    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerDied","Data":"f46eb3a4b5b71bf310e0a8e7ca53143a3ae34eea271aad8a3e73b679e5cd8d6d"}
Nov 01 03:06:24.005536 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:24.005479    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"0ecfb0c06f1429be07e04e979426588bcedaeea1ef3207b0c889bb1c80fa272e"}
Nov 01 03:06:25.843213 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:25.843175    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:25.843213 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:25.843211    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:25.843707 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:25.843295    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:25.843707 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:25.843314    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:25.843707 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:25.843375    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:25.843707 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:25.843431    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:26.012211 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:26.012192    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:06:27.018918 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:27.018891    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:06:27.020300 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:27.020268    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"6cfefbe0a1ce08aae475b598e0064adc63d03a50983a0f54f9b21970af0937d6"}
Nov 01 03:06:27.843451 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:27.843415    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:27.843718 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:27.843486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:27.843718 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:27.843483    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:27.843718 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:27.843574    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:27.843718 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:27.843687    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:27.843883 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:27.843756    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:29.027215 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.027189    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:06:29.027835 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.027599    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"77fbc2af32a044c98f77999e60068c2f0387b5d120a1615082fe83768f6386ac"}
Nov 01 03:06:29.027915 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.027894    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:29.028000 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.027990    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:29.028063 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.028012    2567 scope.go:122] "RemoveContainer" containerID="f46eb3a4b5b71bf310e0a8e7ca53143a3ae34eea271aad8a3e73b679e5cd8d6d"
Nov 01 03:06:29.028063 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.028046    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:29.044750 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.044731    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:29.044835 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.044821    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:06:29.844363 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.844330    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:29.844570 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.844461    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:29.844570 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:29.844477    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:29.844570 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:29.844540    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:29.844570 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:29.844561    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:29.844719 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:29.844647    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:30.032899 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.032872    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:06:30.033299 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.033226    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" event={"ID":"69348615-8e95-479b-b360-6991eff92a2e","Type":"ContainerStarted","Data":"80df5d0aa1ec4bdeb525c62f1444096e055cdec7c3834c6dc3237a6673f87f17"}
Nov 01 03:06:30.786224 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.786110    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct" podStartSLOduration=8.499981158 podStartE2EDuration="29.786090542s" totalImagesPullingTime="21.286109384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:06:30.190969684 +0000 UTC m=+30.870700543" watchObservedRunningTime="2026-11-01 03:06:30.786090542 +0000 UTC m=+31.465821405"
Nov 01 03:06:30.787006 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.786958    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h2vwz"]
Nov 01 03:06:30.787278 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.787257    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:30.787393 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:30.787375    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:30.792879 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.792856    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkdq4"]
Nov 01 03:06:30.792971 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.792947    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:30.793064 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:30.793046    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:30.794593 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.794573    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9r9sn"]
Nov 01 03:06:30.794696 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:30.794673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:30.794799 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:30.794782    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:32.842750 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:32.842715    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:32.843217 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:32.842793    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:32.843217 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:32.842928    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:32.843217 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:32.842963    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:32.843217 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:32.843036    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:32.843217 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:32.843126    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:33.580686 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:33.580647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:33.580856 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.580804    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:33.580901 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.580871    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs podName:02652f44-f471-469d-bf6d-74236195ce03 nodeName:}" failed. No retries permitted until 2026-11-01 03:07:05.580855127 +0000 UTC m=+66.260585965 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs") pod "network-metrics-daemon-9r9sn" (UID: "02652f44-f471-469d-bf6d-74236195ce03") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 03:06:33.681423 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:33.681381    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:33.681584 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:33.681442    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:33.681584 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681543    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:33.681584 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681553    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 03:06:33.681584 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681574    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 03:06:33.681705 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681587    2567 projected.go:196] Error preparing data for projected volume kube-api-access-zzf2x for pod openshift-network-diagnostics/network-check-target-h2vwz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:33.681705 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681598    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret podName:d4321cab-6240-420f-8920-aa78d1f91105 nodeName:}" failed. No retries permitted until 2026-11-01 03:07:05.681584874 +0000 UTC m=+66.361315712 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret") pod "global-pull-secret-syncer-kkdq4" (UID: "d4321cab-6240-420f-8920-aa78d1f91105") : object "kube-system"/"original-pull-secret" not registered
Nov 01 03:06:33.681705 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:33.681629    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x podName:d440d7f0-e153-48ea-94fd-2e5bb181c0a7 nodeName:}" failed. No retries permitted until 2026-11-01 03:07:05.681617814 +0000 UTC m=+66.361348668 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-zzf2x" (UniqueName: "kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x") pod "network-check-target-h2vwz" (UID: "d440d7f0-e153-48ea-94fd-2e5bb181c0a7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 03:06:34.843345 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:34.843263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:34.843345 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:34.843290    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:34.843838 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:34.843369    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9r9sn" podUID="02652f44-f471-469d-bf6d-74236195ce03"
Nov 01 03:06:34.843838 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:34.843263    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:34.843838 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:34.843467    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-h2vwz" podUID="d440d7f0-e153-48ea-94fd-2e5bb181c0a7"
Nov 01 03:06:34.843838 ip-10-0-134-33 kubenswrapper[2567]: E1101 03:06:34.843561    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kkdq4" podUID="d4321cab-6240-420f-8920-aa78d1f91105"
Nov 01 03:06:36.149043 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.149011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d72l5_43635a20-c47b-45c7-9616-8503d796561e/konnectivity-agent/0.log" limit=1024
Nov 01 03:06:36.508994 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.508953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-d72l5_43635a20-c47b-45c7-9616-8503d796561e/konnectivity-agent/0.log" limit=1024
Nov 01 03:06:36.644154 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.644125    2567 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeReady"
Nov 01 03:06:36.644317 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.644245    2567 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 01 03:06:36.854746 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.854660    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:06:36.854746 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.854732    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:06:36.854937 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.854828    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:06:36.919592 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.919560    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 03:06:36.919592 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.919560    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 03:06:36.921845 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.921817    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 03:06:36.922006 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.921970    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-wgrq6\""
Nov 01 03:06:36.922069 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.922056    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-74fv2\""
Nov 01 03:06:36.922111 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.922101    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9gxp\""
Nov 01 03:06:36.922231 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:36.922220    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 03:06:41.021581 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:41.021372    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8ddz9_cb94a967-7b3e-45b4-b117-0b591882dc95/dns-node-resolver/0.log"
Nov 01 03:06:41.825023 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:06:41.824996    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-2lbxr_1a4e4f55-5014-48d0-98cc-669a1cce1dd8/node-ca/0.log"
Nov 01 03:07:01.045590 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:01.045562    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:03.929589 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.929556    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-69f45"]
Nov 01 03:07:03.931898 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.931880    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"]
Nov 01 03:07:03.932051 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.932034    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:03.933359 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.933343    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:03.991358 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.991327    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 01 03:07:03.991536 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.991367    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 01 03:07:03.992052 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.992036    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-585dd\""
Nov 01 03:07:03.996276 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.996259    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtn42\" (UniqueName: \"kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:03.996324 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.996301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:03.996431 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:03.996418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-q29mz\" (UID: \"c2fda3f7-5f7d-401d-9bc7-42116112b337\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:04.000785 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.000768    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-4lwfh\""
Nov 01 03:07:04.000889 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.000795    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 01 03:07:04.037291 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.037264    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s4nk9"]
Nov 01 03:07:04.038898 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.038882    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5ff574547c-4g4gl"]
Nov 01 03:07:04.039021 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.039004    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.042373 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.041506    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9zmr7"]
Nov 01 03:07:04.042628 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.042606    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.044140 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.044116    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.044261 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.044181    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-69f45"]
Nov 01 03:07:04.044750 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.044730    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"]
Nov 01 03:07:04.097150 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097107    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtn42\" (UniqueName: \"kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:04.097150 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097402 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097172    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.097402 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097207    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:04.097402 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097290    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6827n\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097402 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8pm4\" (UniqueName: \"kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.097402 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097613 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.097613 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097483    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.097613 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097502    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:04.097613 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx6vt\" (UniqueName: \"kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.097780 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.097780 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-q29mz\" (UID: \"c2fda3f7-5f7d-401d-9bc7-42116112b337\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:04.097780 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097780 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097780 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097761    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097940 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097788    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.097940 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097810    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.097940 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.097847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.101528 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.101503    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-q29mz\" (UID: \"c2fda3f7-5f7d-401d-9bc7-42116112b337\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:04.109027 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.109007    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 03:07:04.109746 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.109729    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dg4zg\""
Nov 01 03:07:04.109852 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.109839    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8s4ww\""
Nov 01 03:07:04.110135 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.110115    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fdfjz\""
Nov 01 03:07:04.110219 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.110167    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 03:07:04.110219 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.110184    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 03:07:04.110321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.110216    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 03:07:04.110414 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.110400    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 03:07:04.115363 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.115326    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 03:07:04.115363 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.115336    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 03:07:04.115776 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.115756    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 03:07:04.115912 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.115827    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 03:07:04.161091 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.161052    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 03:07:04.161091 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.161088    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 03:07:04.161858 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.161837    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4nk9"]
Nov 01 03:07:04.166848 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.166824    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ff574547c-4g4gl"]
Nov 01 03:07:04.167433 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.167417    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9zmr7"]
Nov 01 03:07:04.167932 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.167918    2567 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-79zbg"]
Nov 01 03:07:04.169685 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.169671    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.180304 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.180246    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtn42\" (UniqueName: \"kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:04.198460 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198467    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198508    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx6vt\" (UniqueName: \"kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198579    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.198638 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198621    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.198937 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198777    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.198937 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198809    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.198937 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198880    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.198937 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198925    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.198955    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199002    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199029    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199061    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4nnd\" (UniqueName: \"kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199114    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.199166 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199139    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.199488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6827n\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199242    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8pm4\" (UniqueName: \"kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.199488 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199481    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.199723 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199697    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.199851 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.199828    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.200181 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.200158    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.200461 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.200439    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.201382 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.201355    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.201537 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.201521    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.201725 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.201701    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.201805 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.201768    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.201941 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.201923    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.211550 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.211500    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79zbg"]
Nov 01 03:07:04.230312 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.230282    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkkrb\""
Nov 01 03:07:04.230467 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.230345    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 03:07:04.231753 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.231736    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 03:07:04.242963 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.242938    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:04.248363 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.248338    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:04.267759 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.267723    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8pm4\" (UniqueName: \"kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.284867 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.284428    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6827n\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.286959 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.286757    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx6vt\" (UniqueName: \"kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.286959 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.286915    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.300751 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.300414    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.300751 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.300455    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.300751 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.300503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4nnd\" (UniqueName: \"kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.300751 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.300523    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.301689 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.301240    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.301689 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.301651    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.303446 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.303423    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.352793 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.352748    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:04.358502 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.358473    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:04.365390 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.365361    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:04.369281 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.369253    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4nnd\" (UniqueName: \"kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.479662 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.479618    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:04.642223 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.642193    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"]
Nov 01 03:07:04.645957 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:04.645918    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2fda3f7_5f7d_401d_9bc7_42116112b337.slice/crio-f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf WatchSource:0}: Error finding container f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf: Status 404 returned error can't find the container with id f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf
Nov 01 03:07:04.693682 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.693532    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-69f45"]
Nov 01 03:07:04.696510 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:04.696475    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d865cd5_54e7_4607_b0a4_a54ca52f61cd.slice/crio-6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1 WatchSource:0}: Error finding container 6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1: Status 404 returned error can't find the container with id 6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1
Nov 01 03:07:04.747833 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.747801    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s4nk9"]
Nov 01 03:07:04.750942 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:04.750908    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda1ddb40_13f0_47ff_be1e_bbc395daf7f6.slice/crio-dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca WatchSource:0}: Error finding container dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca: Status 404 returned error can't find the container with id dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca
Nov 01 03:07:04.760092 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.760064    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ff574547c-4g4gl"]
Nov 01 03:07:04.762245 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.762218    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9zmr7"]
Nov 01 03:07:04.765164 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:04.765069    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650a6586_f42f_4ba9_91b5_375372439c2b.slice/crio-9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb WatchSource:0}: Error finding container 9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb: Status 404 returned error can't find the container with id 9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb
Nov 01 03:07:04.853321 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:04.853179    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-79zbg"]
Nov 01 03:07:04.855851 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:04.855817    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8961e22c_8fdf_4994_9583_d2d1b5a7caa7.slice/crio-8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da WatchSource:0}: Error finding container 8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da: Status 404 returned error can't find the container with id 8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da
Nov 01 03:07:05.112171 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.112129    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-69f45" event={"ID":"0d865cd5-54e7-4607-b0a4-a54ca52f61cd","Type":"ContainerStarted","Data":"6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1"}
Nov 01 03:07:05.113844 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.113814    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl" event={"ID":"65f56877-d0bb-46b0-ab58-560c54fa29f8","Type":"ContainerStarted","Data":"23158946d3283808a9f32923b5c3cb3fcbbf17f082428f3ffac7ee7f5d17ebae"}
Nov 01 03:07:05.114061 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.113862    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl" event={"ID":"65f56877-d0bb-46b0-ab58-560c54fa29f8","Type":"ContainerStarted","Data":"f5f15885a753912211c7e88274fe718195b945c6da51d21adcd8ab1f848bcbf7"}
Nov 01 03:07:05.114061 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.113910    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:05.115075 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.115040    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79zbg" event={"ID":"8961e22c-8fdf-4994-9583-d2d1b5a7caa7","Type":"ContainerStarted","Data":"8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da"}
Nov 01 03:07:05.116370 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.116334    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4nk9" event={"ID":"da1ddb40-13f0-47ff-be1e-bbc395daf7f6","Type":"ContainerStarted","Data":"dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca"}
Nov 01 03:07:05.117948 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.117919    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zmr7" event={"ID":"650a6586-f42f-4ba9-91b5-375372439c2b","Type":"ContainerStarted","Data":"d6637132be2448d905ebfef847435a753f6acfe59a741c39fd7fd3bcd2f26ccd"}
Nov 01 03:07:05.118055 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.117958    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zmr7" event={"ID":"650a6586-f42f-4ba9-91b5-375372439c2b","Type":"ContainerStarted","Data":"9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb"}
Nov 01 03:07:05.119172 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.119152    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz" event={"ID":"c2fda3f7-5f7d-401d-9bc7-42116112b337","Type":"ContainerStarted","Data":"f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf"}
Nov 01 03:07:05.248809 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.248755    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl" podStartSLOduration=2.248741626 podStartE2EDuration="2.248741626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:07:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:05.248284744 +0000 UTC m=+65.928015604" watchObservedRunningTime="2026-11-01 03:07:05.248741626 +0000 UTC m=+65.928472486"
Nov 01 03:07:05.612301 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.612266    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:05.664909 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.664879    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 03:07:05.679949 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.679880    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:05.713151 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.713120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:05.713339 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.713192    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:05.766117 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.766080    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 03:07:05.766286 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.766166    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 03:07:05.778528 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.778342    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:05.803295 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.803246    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9gxp\""
Nov 01 03:07:05.811585 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.811369    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:05.828680 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.828647    2567 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 03:07:05.837960 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:05.837925    2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:06.079599 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.079574    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-wgrq6\""
Nov 01 03:07:06.087070 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.086825    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:06.111606 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.111571    2567 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-74fv2\""
Nov 01 03:07:06.119743 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.119631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:06.128765 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.128732    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zmr7" event={"ID":"650a6586-f42f-4ba9-91b5-375372439c2b","Type":"ContainerStarted","Data":"f3674bf2ba38abf25063b9d7972d83b7c9d231796a5a58ff91441a7215e1e217"}
Nov 01 03:07:06.213364 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.213153    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9r9sn"]
Nov 01 03:07:06.467225 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.467109    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-h2vwz"]
Nov 01 03:07:06.471043 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:06.470969    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd440d7f0_e153_48ea_94fd_2e5bb181c0a7.slice/crio-71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7 WatchSource:0}: Error finding container 71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7: Status 404 returned error can't find the container with id 71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7
Nov 01 03:07:06.501892 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:06.501816    2567 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kkdq4"]
Nov 01 03:07:06.506615 ip-10-0-134-33 kubenswrapper[2567]: W1101 03:07:06.506582    2567 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4321cab_6240_420f_8920_aa78d1f91105.slice/crio-c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9 WatchSource:0}: Error finding container c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9: Status 404 returned error can't find the container with id c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9
Nov 01 03:07:07.134231 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.134125    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kkdq4" event={"ID":"d4321cab-6240-420f-8920-aa78d1f91105","Type":"ContainerStarted","Data":"c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9"}
Nov 01 03:07:07.135856 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.135782    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h2vwz" event={"ID":"d440d7f0-e153-48ea-94fd-2e5bb181c0a7","Type":"ContainerStarted","Data":"71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7"}
Nov 01 03:07:07.138771 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.137735    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz" event={"ID":"c2fda3f7-5f7d-401d-9bc7-42116112b337","Type":"ContainerStarted","Data":"816390d3a1b1a2fc14bb7f2a24840c78bc2413c309e6315886fc90c000a803ba"}
Nov 01 03:07:07.138771 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.138718    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:07.140952 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.140899    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9r9sn" event={"ID":"02652f44-f471-469d-bf6d-74236195ce03","Type":"ContainerStarted","Data":"ede4441da52502e5b7b4cda895f4ed19945892545c2c633e99b248545d3b03bd"}
Nov 01 03:07:07.146048 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.145545    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:07.377077 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:07.376520    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz" podStartSLOduration=2.544343675 podStartE2EDuration="4.376506501s" totalImagesPullingTime="1.832162826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:07.257462045 +0000 UTC m=+67.937192906" watchObservedRunningTime="2026-11-01 03:07:07.376506501 +0000 UTC m=+68.056237353"
Nov 01 03:07:09.152104 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.152065    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79zbg" event={"ID":"8961e22c-8fdf-4994-9583-d2d1b5a7caa7","Type":"ContainerStarted","Data":"95b877d2432eb0149be24a920d80adbc6d4a41f8bd6143d5e098000364d8de7a"}
Nov 01 03:07:09.152593 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.152137    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-79zbg" event={"ID":"8961e22c-8fdf-4994-9583-d2d1b5a7caa7","Type":"ContainerStarted","Data":"e5148458609deeaae6fec43669272b573d2903158f5bccb801e02f6fda6531e7"}
Nov 01 03:07:09.152804 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.152783    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:09.156833 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.156806    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9r9sn" event={"ID":"02652f44-f471-469d-bf6d-74236195ce03","Type":"ContainerStarted","Data":"1ccea061409868a64ad170c3760db4a5c99027d9413ac5b5e2e289217d15dbc7"}
Nov 01 03:07:09.156959 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.156853    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9r9sn" event={"ID":"02652f44-f471-469d-bf6d-74236195ce03","Type":"ContainerStarted","Data":"b67dd60f38ba1a1fc0dcffc6045390dc6fd35cc28e9cd02ee9910c7b3b603fa3"}
Nov 01 03:07:09.279294 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.279236    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-79zbg" podStartSLOduration=1.84945315 podStartE2EDuration="5.279215467s" totalImagesPullingTime="3.429762317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:09.278763168 +0000 UTC m=+69.958494020" watchObservedRunningTime="2026-11-01 03:07:09.279215467 +0000 UTC m=+69.958946328"
Nov 01 03:07:09.403048 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:09.402468    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9r9sn" podStartSLOduration=66.212480906 podStartE2EDuration="1m8.402448813s" totalImagesPullingTime="2.189967907s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:09.401796911 +0000 UTC m=+70.081527775" watchObservedRunningTime="2026-11-01 03:07:09.402448813 +0000 UTC m=+70.082179675"
Nov 01 03:07:10.163889 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:10.163829    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s4nk9" event={"ID":"da1ddb40-13f0-47ff-be1e-bbc395daf7f6","Type":"ContainerStarted","Data":"f860041a38d30f2ef6146427d63b3177c8f6423571ac8aaa2a70a51ae320b5ad"}
Nov 01 03:07:10.168332 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:10.168304    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9zmr7" event={"ID":"650a6586-f42f-4ba9-91b5-375372439c2b","Type":"ContainerStarted","Data":"4c3949976944847dabf66a01e4ed3c7d70012390e9cbd6f88ea995b710cdebff"}
Nov 01 03:07:10.407413 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:10.407349    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s4nk9" podStartSLOduration=2.489996144 podStartE2EDuration="7.407332321s" totalImagesPullingTime="4.917336177s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:10.284162915 +0000 UTC m=+70.963893776" watchObservedRunningTime="2026-11-01 03:07:10.407332321 +0000 UTC m=+71.087063178"
Nov 01 03:07:12.178516 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:12.178474    2567 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-h2vwz" event={"ID":"d440d7f0-e153-48ea-94fd-2e5bb181c0a7","Type":"ContainerStarted","Data":"1a8b9edcbe92c428bb7d99d7831bdc0d05b6b26d66b90d7dce0b38f69e71f4ed"}
Nov 01 03:07:12.179040 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:12.178736    2567 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:12.323642 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:12.323585    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9zmr7" podStartSLOduration=4.941180477 podStartE2EDuration="9.32357155s" totalImagesPullingTime="4.382391073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 03:07:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:10.408524513 +0000 UTC m=+71.088255376" watchObservedRunningTime="2026-11-01 03:07:12.32357155 +0000 UTC m=+73.003302409"
Nov 01 03:07:12.324449 ip-10-0-134-33 kubenswrapper[2567]: I1101 03:07:12.324410    2567 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-h2vwz" podStartSLOduration=66.07605877 podStartE2EDuration="1m11.324402418s" totalImagesPullingTime="5.248343648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:12.322817384 +0000 UTC m=+73.002548245" watchObservedRunningTime="2026-11-01 03:07:12.324402418 +0000 UTC m=+73.004133278"
Nov 01 03:07:13.529227 ip-10-0-134-33 systemd[1]: Stopping Kubernetes Kubelet...
Nov 01 03:07:13.540597 ip-10-0-134-33 systemd[1]: kubelet.service: Deactivated successfully.
Nov 01 03:07:13.540772 ip-10-0-134-33 systemd[1]: Stopped Kubernetes Kubelet.
Nov 01 03:07:13.540936 ip-10-0-134-33 systemd[1]: kubelet.service: Consumed 3.310s CPU time.
Nov 01 03:07:13.554449 ip-10-0-134-33 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 03:07:13.612351 ip-10-0-134-33 kubenswrapper[4892]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:07:13.612351 ip-10-0-134-33 kubenswrapper[4892]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 03:07:13.612351 ip-10-0-134-33 kubenswrapper[4892]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:07:13.612351 ip-10-0-134-33 kubenswrapper[4892]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 03:07:13.614411 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614334    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.614411 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614410    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.614411 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614414    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614417    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614420    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614423    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614426    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614428    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614431    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614434    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614436    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614439    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614442    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614445    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614448    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614450    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614453    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614456    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614459    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614461    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614470    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.614511 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614473    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614476    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614479    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614482    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614485    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614488    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614491    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614494    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614497    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614500    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614502    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614505    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614508    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614511    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614513    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614516    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614519    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614524    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614529    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614533    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.615020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614536    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614538    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614541    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614544    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614546    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614549    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614551    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614554    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614556    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614559    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614562    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614564    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614567    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614569    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614572    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614575    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614578    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614583    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614587    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.615624 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614590    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614592    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614595    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614599    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614602    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614604    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614607    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614609    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614612    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614614    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614617    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614619    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614622    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614625    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614627    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614630    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614633    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614635    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614638    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614640    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.616189 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614643    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614645    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614648    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614650    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614653    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614656    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614658    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614661    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614663    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614666    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614669    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614672    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614674    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614677    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614680    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614683    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614685    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614688    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614690    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614693    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.616703 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614695    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614698    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614701    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614704    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614706    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.614709    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615499    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615507    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615511    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615515    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615518    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615521    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615524    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615527    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615529    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615532    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615536    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615540    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615543    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.617326 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615546    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615549    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615551    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615555    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615557    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615560    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615563    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615566    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615568    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615571    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615574    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615577    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615580    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615582    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615585    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615588    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615590    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615593    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615595    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615599    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.617822 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615601    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615604    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615606    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615609    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615612    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615614    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615617    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615619    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615622    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615625    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615627    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615630    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615633    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615635    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615641    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615645    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615647    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615650    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615653    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.618353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615655    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615658    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615661    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615663    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615666    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615668    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615671    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615674    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615676    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615679    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615681    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615684    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615686    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615692    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615695    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615697    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615700    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615702    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615705    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615707    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.618871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615710    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615714    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615717    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615721    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615726    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615730    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615734    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615740    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615744    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615748    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615752    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615755    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615760    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615764    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615767    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615770    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615772    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615775    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615778    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615781    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.619381 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615784    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615786    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615789    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615791    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615794    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615796    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615800    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615802    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615805    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615808    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615810    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615813    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615815    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.615818    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616346    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616354    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616358    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616360    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616363    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616366    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.619903 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616369    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616371    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616374    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616377    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616379    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616382    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616385    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616387    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616390    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616392    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616395    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616397    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616400    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616402    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616405    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616407    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616410    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616412    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616415    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616418    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.620418 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616420    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616423    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616425    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616428    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616431    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616433    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616436    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616439    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616442    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616444    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616447    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616449    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616452    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616454    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616457    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616459    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616462    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616465    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616467    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.621240 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616470    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616472    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616474    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616477    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616480    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616482    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616485    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616487    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616490    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616492    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616495    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616497    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616500    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616504    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616508    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616512    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616516    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616518    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616521    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616524    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.621849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616527    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616530    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616532    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616535    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616537    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616540    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616542    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616545    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616547    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616549    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616552    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616555    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616557    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616560    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616562    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616564    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616567    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616569    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616588    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616593    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.622372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616596    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616600    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616602    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616605    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616607    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616610    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616613    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616616    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616619    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616621    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616624    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616626    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616629    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616633    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616636    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616640    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616644    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616646    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616650    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.622863 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616653    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.616655    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616746    4892 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616761    4892 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616769    4892 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616776    4892 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616783    4892 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616787    4892 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616793    4892 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616798    4892 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616801    4892 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616804    4892 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616808    4892 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616811    4892 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616814    4892 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616817    4892 flags.go:64] FLAG: --cgroup-root=""
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616820    4892 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616823    4892 flags.go:64] FLAG: --client-ca-file=""
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616827    4892 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616830    4892 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616834    4892 flags.go:64] FLAG: --cluster-domain=""
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616837    4892 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616840    4892 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616843    4892 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 03:07:13.623402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616847    4892 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616851    4892 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616855    4892 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616858    4892 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616862    4892 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616865    4892 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616868    4892 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616871    4892 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616874    4892 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616877    4892 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616882    4892 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616885    4892 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616889    4892 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616891    4892 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616894    4892 flags.go:64] FLAG: --enable-server="true"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616897    4892 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616902    4892 flags.go:64] FLAG: --event-burst="100"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616906    4892 flags.go:64] FLAG: --event-qps="50"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616909    4892 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616912    4892 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616915    4892 flags.go:64] FLAG: --eviction-hard=""
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616919    4892 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616922    4892 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616925    4892 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616928    4892 flags.go:64] FLAG: --eviction-soft=""
Nov 01 03:07:13.624066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616931    4892 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616934    4892 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616937    4892 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616941    4892 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616944    4892 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616947    4892 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616950    4892 flags.go:64] FLAG: --feature-gates=""
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616954    4892 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616957    4892 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616962    4892 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616965    4892 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616968    4892 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616971    4892 flags.go:64] FLAG: --help="false"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616987    4892 flags.go:64] FLAG: --hostname-override="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616990    4892 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616994    4892 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.616997    4892 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617000    4892 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617003    4892 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617006    4892 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617009    4892 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617012    4892 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617015    4892 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 03:07:13.624691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617020    4892 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617023    4892 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617026    4892 flags.go:64] FLAG: --kube-reserved=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617029    4892 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617033    4892 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617036    4892 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617038    4892 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617041    4892 flags.go:64] FLAG: --lock-file=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617044    4892 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617047    4892 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617050    4892 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617055    4892 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617058    4892 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617062    4892 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617065    4892 flags.go:64] FLAG: --logging-format="text"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617068    4892 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617071    4892 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617074    4892 flags.go:64] FLAG: --manifest-url=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617078    4892 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617082    4892 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617085    4892 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617089    4892 flags.go:64] FLAG: --max-pods="110"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617092    4892 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617096    4892 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617099    4892 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 03:07:13.625261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617102    4892 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617105    4892 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617108    4892 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617110    4892 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617117    4892 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617120    4892 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617123    4892 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617128    4892 flags.go:64] FLAG: --pod-cidr=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617131    4892 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617134    4892 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617137    4892 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617141    4892 flags.go:64] FLAG: --port="10250"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617144    4892 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617147    4892 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0f20d9680c708fe1c"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617150    4892 flags.go:64] FLAG: --qos-reserved=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617153    4892 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617156    4892 flags.go:64] FLAG: --register-node="true"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617159    4892 flags.go:64] FLAG: --register-with-taints=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617162    4892 flags.go:64] FLAG: --registry-burst="10"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617165    4892 flags.go:64] FLAG: --registry-qps="5"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617168    4892 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617171    4892 flags.go:64] FLAG: --reserved-memory=""
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617175    4892 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617178    4892 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617181    4892 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617186    4892 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 03:07:13.626056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617189    4892 flags.go:64] FLAG: --runonce="false"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617192    4892 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617195    4892 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617198    4892 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617200    4892 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617204    4892 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617207    4892 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617210    4892 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617213    4892 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617216    4892 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617219    4892 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617222    4892 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617225    4892 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617230    4892 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617233    4892 flags.go:64] FLAG: --system-cgroups=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617235    4892 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617241    4892 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617244    4892 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617246    4892 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617250    4892 flags.go:64] FLAG: --tls-min-version=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617253    4892 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617256    4892 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617259    4892 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617262    4892 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617265    4892 flags.go:64] FLAG: --v="2"
Nov 01 03:07:13.626694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617269    4892 flags.go:64] FLAG: --version="false"
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617273    4892 flags.go:64] FLAG: --vmodule=""
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617277    4892 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617281    4892 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617415    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617418    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617423    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617426    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617430    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617432    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617435    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617438    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617441    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617444    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617446    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617449    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617451    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617454    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617457    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617461    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.627370 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617466    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617469    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617473    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617475    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617479    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617482    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617485    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617488    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617491    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617493    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617495    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617498    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617501    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617503    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617506    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617508    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617512    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617515    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617519    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.627939 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617522    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617524    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617527    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617530    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617533    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617535    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617538    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617540    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617543    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617545    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617548    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617550    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617553    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617557    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617560    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617563    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617575    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617577    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617580    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617583    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.628591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617585    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617588    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617590    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617593    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617596    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617598    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617601    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617603    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617606    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617608    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617613    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617618    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617621    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617624    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617626    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617629    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617632    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617635    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617638    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617640    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.629113 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617643    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617645    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617648    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617651    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617654    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617658    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617660    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617663    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617665    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617668    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617671    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617673    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617676    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617679    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617681    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617683    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617686    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617689    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617691    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.629718 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617694    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617696    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617699    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617702    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617706    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617709    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617711    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617714    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617716    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617719    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617721    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617724    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.617750    4892 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617788    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617793    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617795    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.630335 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617798    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617801    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617804    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617808    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617811    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617813    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617816    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617818    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617821    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617823    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617826    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617829    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617832    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617835    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617838    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617840    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617843    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617845    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617848    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617851    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.630781 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617854    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617857    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617860    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617863    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617865    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617868    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617870    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617873    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617875    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617878    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617882    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617885    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617888    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617891    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617893    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617898    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617900    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617903    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617906    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617908    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.631319 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617911    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617914    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617916    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617919    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617921    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617924    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617926    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617929    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617932    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617934    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617937    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617939    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617944    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617946    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617949    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617952    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617954    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617957    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617959    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617962    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.631815 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617965    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617968    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617971    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617974    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617988    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617991    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617993    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.617997    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618000    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618003    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618006    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618008    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618011    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618013    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618016    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618019    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618021    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618024    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618026    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618029    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.632430 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618032    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618034    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618037    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618039    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618044    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618049    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618052    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618056    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618059    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618062    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618065    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618068    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618070    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618073    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618075    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618078    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618080    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618083    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618085    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.632963 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618089    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618092    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618095    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.618097    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.622769    4892 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.622785    4892 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.622800    4892 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.622805    4892 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622861    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622866    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622870    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622873    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622876    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622879    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622882    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622885    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622887    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622891    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622895    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.633446 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622899    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622902    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622904    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622907    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622910    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622912    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622916    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622920    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622924    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622928    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622931    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622933    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622937    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622940    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622942    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622945    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622948    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622951    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622954    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.633955 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622957    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622960    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622963    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622966    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622968    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622971    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622974    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.622997    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623001    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623004    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623007    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623010    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623012    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623015    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623018    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623020    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623023    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623026    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623029    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.634451 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623033    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623035    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623038    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623041    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623043    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623046    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623049    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623051    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623054    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623057    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623059    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623063    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623065    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623068    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623071    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623073    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623076    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623078    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623081    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.634946 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623084    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623086    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623089    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623091    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623094    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623097    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623099    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623102    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623105    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623108    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623111    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623114    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623117    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623120    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623123    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623125    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623128    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623130    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623133    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623136    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.635569 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623139    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623141    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623144    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623147    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623149    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623152    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623155    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623157    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623160    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623163    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623165    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623168    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623171    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623173    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623176    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623179    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623182    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.636291 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623184    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.623213    4892 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623367    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623373    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623377    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623381    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623384    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623388    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623390    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623393    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623396    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623398    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623401    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623404    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623407    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623410    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.636872 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623412    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623415    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623418    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623421    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623424    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623427    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623429    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623432    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623434    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623437    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623440    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623442    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623445    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623448    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623450    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623453    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623455    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623458    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623461    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623463    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.637507 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623466    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623468    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623471    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623474    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623477    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623480    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623482    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623485    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623488    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623490    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623493    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623495    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623498    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623500    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623503    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623505    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623508    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623510    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623513    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.638325 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623515    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623518    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623520    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623523    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623526    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623528    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623531    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623534    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623536    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623539    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623541    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623544    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623547    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623549    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623552    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623554    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623557    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623560    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623563    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623566    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.639111 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623568    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623571    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623573    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623576    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623578    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623581    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623583    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623586    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623588    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623591    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623595    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623598    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623601    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623604    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623606    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623609    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623612    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623614    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623617    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.639606 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623619    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623622    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623624    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623627    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623629    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623632    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623634    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623637    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623639    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623642    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623645    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623647    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623650    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623652    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.623676    4892 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 03:07:13.640406 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623706    4892 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623710    4892 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623712    4892 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623715    4892 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623718    4892 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623720    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623723    4892 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623725    4892 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623728    4892 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623730    4892 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623733    4892 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623735    4892 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623738    4892 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623741    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623744    4892 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623746    4892 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623749    4892 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623752    4892 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623754    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 03:07:13.641024 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623757    4892 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623759    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623762    4892 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623765    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623767    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623770    4892 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623773    4892 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623775    4892 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623778    4892 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623780    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623783    4892 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623786    4892 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623789    4892 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623791    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623794    4892 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623796    4892 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623799    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623801    4892 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623804    4892 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623806    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 03:07:13.641861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623809    4892 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623811    4892 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623814    4892 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623817    4892 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623819    4892 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623822    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623824    4892 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623827    4892 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623829    4892 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623832    4892 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623834    4892 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623837    4892 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623839    4892 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623842    4892 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623845    4892 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623849    4892 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623853    4892 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623856    4892 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623859    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 03:07:13.642443 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623861    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623864    4892 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623866    4892 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623869    4892 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623871    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623874    4892 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623877    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623879    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623882    4892 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623884    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623887    4892 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623889    4892 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623892    4892 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623894    4892 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623897    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623900    4892 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623902    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623905    4892 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623907    4892 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 03:07:13.642932 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623910    4892 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623913    4892 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623917    4892 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623919    4892 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623922    4892 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623925    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623928    4892 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623930    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623933    4892 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623936    4892 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623938    4892 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623941    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623944    4892 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623946    4892 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623949    4892 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623952    4892 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623955    4892 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623957    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623960    4892 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623963    4892 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 03:07:13.643415 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623966    4892 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623968    4892 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623971    4892 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623973    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623990    4892 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.623997    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.624000    4892 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.624002    4892 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:13.624005    4892 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.624057    4892 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.624960    4892 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625039    4892 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625234    4892 server.go:1014] "Starting client certificate rotation"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625393    4892 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625448    4892 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 03:01:00 +0000 UTC" deadline="2028-07-08 01:37:33.103110092 +0000 UTC"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625490    4892 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14758h30m19.47762386s"
Nov 01 03:07:13.643901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.625726    4892 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 03:07:13.644397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.626778    4892 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 03:07:13.644397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.627852    4892 log.go:25] "Validated CRI v1 runtime API"
Nov 01 03:07:13.644397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.629523    4892 log.go:25] "Validated CRI v1 image API"
Nov 01 03:07:13.644397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.630305    4892 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 03:07:13.644397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.631955    4892 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 98f1df92-0359-4698-8cf1-205e8d803fea:/dev/nvme0n1p3 a7fb9b66-8ff1-4406-a7e8-bfcf68ed423e:/dev/nvme0n1p4]
Nov 01 03:07:13.644545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.631972    4892 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825/userdata/shm major:0 minor:85 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7/userdata/shm major:0 minor:255 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da/userdata/shm major:0 minor:222 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb/userdata/shm major:0 minor:220 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d/userdata/shm major:0 minor:71 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9/userdata/shm major:0 minor:260 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca/userdata/shm major:0 minor:215 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ede4441da52502e5b7b4cda895f4ed19945892545c2c633e99b248545d3b03bd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ede4441da52502e5b7b4cda895f4ed19945892545c2c633e99b248545d3b03bd/userdata/shm major:0 minor:250 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f5f15885a753912211c7e88274fe718195b945c6da51d21adcd8ab1f848bcbf7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f5f15885a753912211c7e88274fe718195b945c6da51d21adcd8ab1f848bcbf7/userdata/shm major:0 minor:217 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/00459c43-88b4-494f-a76f-3cbef932a916/volumes/kubernetes.io~projected/kube-api-access-ttw5v:{mountpoint:/var/lib/kubelet/pods/00459c43-88b4-494f-a76f-3cbef932a916/volumes/kubernetes.io~projected/kube-api-access-ttw5v major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~projected/kube-api-access-pfwbf:{mountpoint:/var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~projected/kube-api-access-pfwbf major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~secret/metrics-certs major:0 minor:248 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d865cd5-54e7-4607-b0a4-a54ca52f61cd/volumes/kubernetes.io~projected/kube-api-access-rtn42:{mountpoint:/var/lib/kubelet/pods/0d865cd5-54e7-4607-b0a4-a54ca52f61cd/volumes/kubernetes.io~projected/kube-api-access-rtn42 major:0 minor:200 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/18679b6e-26b2-4b4d-8169-d30c6e45d201/volumes/kubernetes.io~projected/kube-api-access-pmqwm:{mountpoint:/var/lib/kubelet/pods/18679b6e-26b2-4b4d-8169-d30c6e45d201/volumes/kubernetes.io~projected/kube-api-access-pmqwm major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a4e4f55-5014-48d0-98cc-669a1cce1dd8/volumes/kubernetes.io~projected/kube-api-access-wc5h9:{mountpoint:/var/lib/kubelet/pods/1a4e4f55-5014-48d0-98cc-669a1cce1dd8/volumes/kubernetes.io~projected/kube-api-access-wc5h9 major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/tmp major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~projected/kube-api-access-fsrhz:{mountpoint:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~projected/kube-api-access-fsrhz major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2bc70ed0-1987-4933-9c27-e98b5b15dc32/volumes/kubernetes.io~projected/kube-api-access-wbrgx:{mountpoint:/var/lib/kubelet/pods/2bc70ed0-1987-4933-9c27-e98b5b15dc32/volumes/kubernetes.io~projected/kube-api-access-wbrgx major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/43635a20-c47b-45c7-9616-8503d796561e/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/43635a20-c47b-45c7-9616-8503d796561e/volumes/kubernetes.io~secret/agent-certs major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~projected/kube-api-access-xx6vt:{mountpoint:/var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~projected/kube-api-access-xx6vt major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/kube-api-access-6827n:{mountpoint:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/kube-api-access-6827n major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/registry-tls major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~projected/kube-api-access-rqpfb:{mountpoint:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~projected/kube-api-access-rqpfb major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~projected/kube-api-access-j4nnd:{mountpoint:/var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~projected/kube-api-access-j4nnd major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~secret/metrics-tls major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b2e9266-2c96-49c3-8d2f-763aeb410cd2/volumes/kubernetes.io~projected/kube-api-access-pxkth:{mountpoint:/var/lib/kubelet/pods/8b2e9266-2c96-49c3-8d2f-763aeb410cd2/volumes/kubernetes.io~projected/kube-api-access-pxkth major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c2fda3f7-5f7d-401d-9bc7-42116112b337/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/c2fda3f7-5f7d-401d-9bc7-42116112b337/volumes/kubernetes.io~secret/tls-certificates major:0 minor:196 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb94a967-7b3e-45b4-b117-0b591882dc95/volumes/kubernetes.io~projected/kube-api-access-2rlhw:{mountpoint:/var/lib/kubelet/pods/cb94a967-7b3e-45b4-b117-0b591882dc95/volumes/kubernetes.io~projected/kube-api-access-2rlhw major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d4321cab-6240-420f-8920-aa78d1f91105/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/d4321cab-6240-420f-8920-aa78d1f91105/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:249 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d440d7f0-e153-48ea-94fd-2e5bb181c0a7/volumes/kubernetes.io~projected/kube-api-access-zzf2x:{mountpoint:/var/lib/kubelet/pods/d440d7f0-e153-48ea-94fd-2e5bb181c0a7/volumes/kubernetes.io~projected/kube-api-access-zzf2x major:0 minor:252 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~projected/kube-api-access-m8pm4:{mountpoint:/var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~projected/kube-api-access-m8pm4 major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/2a7ec7e1a033c7899da35fcb19a0ba173b2555728809d863e44879af5bda966c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/aec5561901541b1b3206d47bc5e627bac72c14a70eb9ffb0d4af9df4d39fb840/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/0c41e172e6eeefb39db6530aeddd7dec24f072fca18d425b91556eeb5bd81010/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/9100912a4ba1eeff80aaf13144b2b4fcd746c15883f84870ad17a0b90ab6b17c/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/ca952d93bf2d5a393d6ccb910ddf9c5a6b3fc0c1d35ff784f62cba163bd16158/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/10fb7d1382f7e59d6328e198155d193f2c33bed1b0be9f9eeb4c1abb859d1d77/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/457066b7d94fecbedfe15ca034781985a8ece837b83294a2b2361851db023e85/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/e0fa82f3a33fd058be567da94ab6fddc03852ab778ec51593a4f31b120577248/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/e81f909438dd20a8592a982727682b6f65b90c856d0b53b79f290eade5e0543d/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-131:{mountpoint:/var/lib/containers/storage/overlay/f8675283339cb280cbc1e35f8221e712ec7b0d5602d79d32e141876e45c545e7/merged major:0 minor:131 fsType:overlay blockSize:0} overlay_0-138:{mountpoint:/var/lib/containers/storage/overlay/3ce8a15be289431944e23831c00c467690c0ed225fa12b2c52bac582cfd37110/merged major:0 minor:138 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/e806bcd7bf8cb40b5ee20fb48d3c7fb49da973052ba44ad14cce2ea1ad8ee937/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/75cc79cc76cf004cf79efe0dac0a0427670f9d92bac05171fab452899f79b19f/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/deafb9a6ec3a00ca94aef5b3eb4400263ea40e36d6c05bc15e3239c19ecbd91f/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/8bb37cbaf77384c2f1d51bf96fa9379eb932c7ca5564d6d7fa13a82ccd6548f0/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/e5df995ec8dff1e3b0f171ecde23b5368256a28788b08857ecb1da11b5da3a6c/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/564e6f85a77f57dc008a9dfb370da6aade1a7a0d2323c11d7957361017569d6a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/7d33cefdb1dd3a9211edea2ecc7ec8f849914e471d7ba5dffdc511cbced13919/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/3d61b3739ab53dbffbad824b9237d32101b013963cb28d35ef958fc031103123/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/b598475ff79823169e9d6c302e940ba99ee51587608deaf8ec6bca310ab02c06/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-224:{mountpoint:/var/lib/containers/storage/overlay/a446aa0e0e874c69ed732ac6898abdad6f5ad1a97a1d02f5d8ebdc690d60d9a9/merged major:0 minor:224 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/c7a8b602cddcffa3a80a5501c1a7e39120ac62647c0ee5ab55633d6c5cd1cc4b/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/3d5788bd7bbb9f72b4cdbf2084d59ffccf929a15b298df0c12f5053ecb2e3a1d/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/968961f7225e68e9b84a0886b4ffd76ed4c42d4f90798e7c0c496cff511cd577/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/b48556c87ac04d3d8459978c3e1f6ac6f84d0e716299511d6f55031cefd06e32/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-234:{mountpoint:/var/lib/containers/storage/overlay/6ba6b70cfefe027eb0a83f73a4b89a478b8421be0ccbf49b29259c5d70a17f19/merged major:0 minor:234 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/1c0f784ef22edb3726778c85a4a47659b34d11a6c695efeea9b2be35eaaea3e5/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-238:{mountpoint:/var/lib/containers/storage/overlay/bd4454de245f8eb5533eac45c3f8722dc1d6deedd21d2901bc2957a74e78d691/merged major:0 minor:238 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/3ea5d3a622e1d9186e549e46ef5286c6e2064337a07db38650c5337b2711d167/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/cea806ef46b1e0a1c3091e20a46c25f7b8540f3367fce7054a52f6b11571750d/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/5d3306d23adb5c1320e87c711e145771e31f70e20fbed6b893f419300a5f5e03/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/535cf6f2cba3efa91686c2fe17bf0b9f07d7aa0126a63ab1fabcc67a3289d8dc/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/162bf6eea170e6d5f58bebbb39addd450a8fda18c979b8300d48bcb3c19e6fc7/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/a2d7f1aeb9384b07e9ede6cc6baa8722b0f772ed61f8fc7e6be4d94925085223/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/aa5dddeb721f88738d70d38f11037fe0ffa54af7c2e569b8d97f6e3d108fdde3/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/ec2f924a336495a4ec022c56eb83c1cf61bd14a73fe2a0f5d94b16726b092549/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-284:{mountpoint:/var/lib/containers/storage/overlay/f3a7934fb2032e71b8eab99fee188acc3069c9c57c86c38c4bbd68ed8055b997/merged major:0 minor:284 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/71d63d604b4ddea2c5d4d97ba13a3f7a5601edfec145397e1ed718061395f9fa/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/7f24b633d50d24f11e5442a07d4035cf21be95e8dd107ae3740e8f94f5a04d72/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/4bd6f1babb16a71c3029aa1d416df0a3cf803c768dbcd524933c74d4832d04e4/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/387f70415b3c324dac95e2b7c5b171ebd9ae84cb9a6873613bc5226a73ef97f5/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-45:{mountpoint:/var/lib/containers/storage/overlay/f5be88fe723f7fbee73d94f04cf4bd4b8bfc10730c774fd6728544bba7e90e07/merged major:0 minor:45 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/d47bb2d2f7ddf8be7e296c28adbfd53022eaba75b1ce783edcf9b050f7e4ef44/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/f1dec044a73c4389b6a970257668a64453e2c85f107dcd92f6f4bb98212ca211/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/dc3b9f0478bf5374235799bfe5ab202ff1f12fca9c55e0cd7ec8353ec766b8f9/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/1afb2c1049dcacfc7f41fb1752a480a4b6f5fbb92fbce0193187bc668b90a93a/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/370409ceb00ddba14881758e6377f6b353475b89da934f68ce47454c19d08297/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/a54ecaa61b457d9884bfa93599371d0e4374acaa348819fe8cdf88e1dd634443/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/5f53a5d462bcf582a91d35490287169ffd672ea8ae53555095dd191fc74c6d09/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/790c3693a0df3cf28fc719adeadd3b24d975680c2e75a48c7e1dfe5d6a4242bc/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/8b2c80bacf3ee1bd0ef554e170fc7addbb2edc2765b9c22ef4af9233106d682f/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/cbd26c19a5cef74ffadc983afe9ede4bdb015c92053e3f6acceee8dd82816c81/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.639260    4892 manager.go:217] Machine: {Timestamp:2026-11-01 03:07:13.638500076 +0000 UTC m=+0.054399599 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113501 MemoryCapacity:33164324864 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c706d71ed0668ba079ff590c42da7 SystemUUID:ec2c706d-71ed-0668-ba07-9ff590c42da7 BootID:728bde97-8cda-4083-a5ea-0beafdcac73d Filesystems:[{Device:/run/containers/storage/overlay-containers/8b879e2fdb9d9ae9b0a6b05ba2a260b59b0e36c9e8f6aedcddeceec1423172da/userdata/shm DeviceMajor:0 DeviceMinor:222 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-284 DeviceMajor:0 DeviceMinor:284 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/540b887d57179b74ef0d8f03fc44faae5883f70b18c266ee3e282c43ad015825/userdata/shm DeviceMajor:0 DeviceMinor:85 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/d440d7f0-e153-48ea-94fd-2e5bb181c0a7/volumes/kubernetes.io~projected/kube-api-access-zzf2x DeviceMajor:0 DeviceMinor:252 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~projected/kube-api-access-fsrhz DeviceMajor:0 DeviceMinor:59 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/00459c43-88b4-494f-a76f-3cbef932a916/volumes/kubernetes.io~projected/kube-api-access-ttw5v DeviceMajor:0 DeviceMinor:62 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c2fda3f7-5f7d-401d-9bc7-42116112b337/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:196 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c8323805698c6977097ef175ede54076e7af95b52c48d805a9248e5137b9c1f9/userdata/shm DeviceMajor:0 DeviceMinor:260 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volumes/kubernetes.io~projected/kube-api-access-rqpfb DeviceMajor:0 DeviceMinor:61 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:204 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93779a61020732816131feafe4347eb6ff523c46f309f6e98f18fe70767672f6/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:213 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/43635a20-c47b-45c7-9616-8503d796561e/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:63 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~projected/kube-api-access-m8pm4 DeviceMajor:0 DeviceMinor:210 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:205 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/71d7e90dd110a672c20c8ca3d08e6fc9584c3b79062e4c3f3d68e858517562a7/userdata/shm DeviceMajor:0 DeviceMinor:255 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d5713749ddf7de5bb20298f46d5140c5acb5afdbe9f0772516c33fd4af6880c5/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-238 DeviceMajor:0 DeviceMinor:238 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2bc70ed0-1987-4933-9c27-e98b5b15dc32/volumes/kubernetes.io~projected/kube-api-access-wbrgx DeviceMajor:0 DeviceMinor:64 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/18679b6e-26b2-4b4d-8169-d30c6e45d201/volumes/kubernetes.io~projected/kube-api-access-pmqwm DeviceMajor:0 DeviceMinor:67 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:203 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6e09d008f03472c00d96db4080761eefdfc1b1bb34db6125ec03ab15c66619b1/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:214 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f5f15885a753912211c7e88274fe718195b945c6da51d21adcd8ab1f848bcbf7/userdata/shm DeviceMajor:0 DeviceMinor:217 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c7396823980cc3daa1bf791e0ebd5b2da1a7748201b2e178178e4d0084fcacb/userdata/shm DeviceMajor:0 DeviceMinor:220 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582160384 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~projected/kube-api-access-pfwbf DeviceMajor:0 DeviceMinor:58 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:202 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a4e4f55-5014-48d0-98cc-669a1cce1dd8/volumes/kubernetes.io~projected/kube-api-access-wc5h9 DeviceMajor:0 DeviceMinor:66 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/da1ddb40-13f0-47ff-be1e-bbc395daf7f6/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/650a6586-f42f-4ba9-91b5-375372439c2b/volumes/kubernetes.io~projected/kube-api-access-xx6vt DeviceMajor:0 DeviceMinor:212 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-224 DeviceMajor:0 DeviceMinor:224 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cb94a967-7b3e-45b4-b117-0b591882dc95/volumes/kubernetes.io~projected/kube-api-access-2rlhw DeviceMajor:0 DeviceMinor:60 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9bab9c5f11c5b27dd7098b2a0451fb6e5f8f1d68d974d9e82b40621cfff8db00/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f46f45ab70452ff898151ba48414d572ca56cc476de31c9f79a8182fa5f811cf/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/65f56877-d0bb-46b0-ab58-560c54fa29f8/volumes/kubernetes.io~projected/kube-api-access-6827n DeviceMajor:0 DeviceMinor:211 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-131 DeviceMajor:0 DeviceMinor:131 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8961e22c-8fdf-4994-9583-d2d1b5a7caa7/volumes/kubernetes.io~projected/kube-api-access-j4nnd DeviceMajor:0 DeviceMinor:219 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-234 DeviceMajor:0 DeviceMinor:234 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d4321cab-6240-420f-8920-aa78d1f91105/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:249 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/be7a788e0c1a1b1ca35bc2eda2c1976f4f0e300d1a2e17ff086aeb577ab2213d/userdata/shm DeviceMajor:0 DeviceMinor:71 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/01e7069b1b2c85933d27fbfb23ae360f8c27ceda1d39c87b6179561871948037/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/04284accff0e81de0cae34e7f8ca813794636a11866a81ce67a504628d969dc8/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02652f44-f471-469d-bf6d-74236195ce03/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:248 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ede4441da52502e5b7b4cda895f4ed19945892545c2c633e99b248545d3b03bd/userdata/shm DeviceMajor:0 DeviceMinor:250 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-45 DeviceMajor:0 DeviceMinor:45 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/26425cb6-d11f-4434-ba47-732c8fe23280/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:55 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/8b2e9266-2c96-49c3-8d2f-763aeb410cd2/volumes/kubernetes.io~projected/kube-api-access-pxkth DeviceMajor:0 DeviceMinor:65 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/69348615-8e95-479b-b360-6991eff92a2e/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-138 DeviceMajor:0 DeviceMinor:138 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc697c22e6625d69be26cdc233c45da126a1892d442d38e235d1715d6d74aba3/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true} {Device:/var/lib/kubelet/pods/0d865cd5-54e7-4607-b0a4-a54ca52f61cd/volumes/kubernetes.io~projected/kube-api-access-rtn42 DeviceMajor:0 DeviceMinor:200 Capacity:31985725440 Type:vfs Inodes:4048379 HasInodes:true} {Device:/run/containers/storage/overlay-containers/dfa626b92ded44f4c67712823c738b355249a8ee47e79ccc46da1c40e1030aca/userdata/shm DeviceMajor:0 DeviceMinor:215 Capacity:67108864 Type:vfs Inodes:4048379 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:6e09d008f03472c MacAddress:b2:4e:60:4c:1c:8b Speed:10000 Mtu:8901} {Name:71d7e90dd110a67 MacAddress:ba:8c:f5:c9:c9:ed Speed:10000 Mtu:8901} {Name:8b879e2fdb9d9ae MacAddress:f6:1b:46:24:f5:03 Speed:10000 Mtu:8901} {Name:9c7396823980cc3 MacAddress:be:f9:b5:54:4d:dd Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:7e:5d:63:10:ab Speed:0 Mtu:9001} {Name:br-int MacAddress:8e:79:6f:fb:a0:e7 Speed:0 Mtu:8901} {Name:c8323805698c697 MacAddress:06:9e:c9:0f:98:a8 Speed:10000 Mtu:8901} {Name:dfa626b92ded44f MacAddress:c2:70:cb:71:9d:ba Speed:10000 Mtu:8901} {Name:ede4441da52502e MacAddress:9a:21:ff:49:a5:44 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:7e:5d:63:10:ab Speed:0 Mtu:9001} {Name:f46f45ab70452ff MacAddress:ea:95:37:b1:38:97 Speed:10000 Mtu:8901} {Name:f5f15885a753912 MacAddress:52:3a:85:3f:0c:b7 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:12:02:a2:71:f5:7f Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:42:25:61:13:cd:19 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164324864 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.639540    4892 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.639582    4892 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.639922    4892 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.639957    4892 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-33.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640163    4892 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640176    4892 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640190    4892 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640208    4892 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640334    4892 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640548    4892 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640689    4892 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640734    4892 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640754    4892 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640768    4892 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640781    4892 kubelet.go:405] "Adding apiserver pod source"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.640792    4892 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.641609    4892 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.641623    4892 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 03:07:13.644719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.642138    4892 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 03:07:13.652956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.652942    4892 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 03:07:13.652956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.652956    4892 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 03:07:13.653072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653062    4892 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 01 03:07:13.653250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653240    4892 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 03:07:13.653281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653269    4892 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 03:07:13.653372 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653362    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 03:07:13.653402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653378    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 03:07:13.653402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653385    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 03:07:13.653402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653391    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 03:07:13.653402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653396    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 03:07:13.653402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653402    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653408    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653413    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653420    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653426    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653434    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653444    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653469    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 03:07:13.653566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.653474    4892 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 03:07:13.656676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.656662    4892 server.go:1263] "Started kubelet"
Nov 01 03:07:13.656775 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.656709    4892 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 03:07:13.656828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.656756    4892 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 03:07:13.657963 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.656847    4892 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 03:07:13.657963 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.657228    4892 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 03:07:13.657246 ip-10-0-134-33 systemd[1]: Started Kubernetes Kubelet.
Nov 01 03:07:13.658349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.658279    4892 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 03:07:13.660803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.660605    4892 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 03:07:13.661259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661199    4892 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 03:07:13.661431 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.661410    4892 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:07:13.661431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661412    4892 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 03:07:13.661625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661410    4892 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 03:01:02 +0000 UTC" deadline="2028-06-10 22:15:00.756824405 +0000 UTC"
Nov 01 03:07:13.661625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661452    4892 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14107h7m47.095376145s"
Nov 01 03:07:13.661625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661429    4892 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 03:07:13.661625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.661441    4892 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 03:07:13.663412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.663391    4892 factory.go:55] Registering systemd factory
Nov 01 03:07:13.663509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.663418    4892 factory.go:223] Registration of the systemd container factory successfully
Nov 01 03:07:13.664089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664070    4892 factory.go:153] Registering CRI-O factory
Nov 01 03:07:13.664171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664093    4892 factory.go:223] Registration of the crio container factory successfully
Nov 01 03:07:13.664171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664147    4892 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 03:07:13.664171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664165    4892 factory.go:103] Registering Raw factory
Nov 01 03:07:13.664357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664178    4892 manager.go:1199] Started watching for new ooms in manager
Nov 01 03:07:13.664421 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664405    4892 manager.go:319] Starting recovery of all containers
Nov 01 03:07:13.665386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.664057    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26425cb6-d11f-4434-ba47-732c8fe23280" volumeName="kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz" seLinuxMountContext=""
Nov 01 03:07:13.665454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665397    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="00459c43-88b4-494f-a76f-3cbef932a916" volumeName="kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v" seLinuxMountContext=""
Nov 01 03:07:13.665454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665417    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18679b6e-26b2-4b4d-8169-d30c6e45d201" volumeName="kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 01 03:07:13.665454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665430    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca" seLinuxMountContext=""
Nov 01 03:07:13.665454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665443    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8961e22c-8fdf-4994-9583-d2d1b5a7caa7" volumeName="kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665456    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d440d7f0-e153-48ea-94fd-2e5bb181c0a7" volumeName="kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665469    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d865cd5-54e7-4607-b0a4-a54ca52f61cd" volumeName="kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665483    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="43635a20-c47b-45c7-9616-8503d796561e" volumeName="kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665499    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="43635a20-c47b-45c7-9616-8503d796561e" volumeName="kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665511    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69348615-8e95-479b-b360-6991eff92a2e" volumeName="kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665524    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02652f44-f471-469d-bf6d-74236195ce03" volumeName="kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665536    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d865cd5-54e7-4607-b0a4-a54ca52f61cd" volumeName="kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665549    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18679b6e-26b2-4b4d-8169-d30c6e45d201" volumeName="kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy" seLinuxMountContext=""
Nov 01 03:07:13.665571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665564    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4e4f55-5014-48d0-98cc-669a1cce1dd8" volumeName="kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665583    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b2e9266-2c96-49c3-8d2f-763aeb410cd2" volumeName="kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665596    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18679b6e-26b2-4b4d-8169-d30c6e45d201" volumeName="kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665608    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69348615-8e95-479b-b360-6991eff92a2e" volumeName="kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665621    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69348615-8e95-479b-b360-6991eff92a2e" volumeName="kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665634    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da1ddb40-13f0-47ff-be1e-bbc395daf7f6" volumeName="kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665648    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665660    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02652f44-f471-469d-bf6d-74236195ce03" volumeName="kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665673    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="18679b6e-26b2-4b4d-8169-d30c6e45d201" volumeName="kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665685    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665697    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb94a967-7b3e-45b4-b117-0b591882dc95" volumeName="kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665712    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a4e4f55-5014-48d0-98cc-669a1cce1dd8" volumeName="kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665725    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bc70ed0-1987-4933-9c27-e98b5b15dc32" volumeName="kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665738    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bc70ed0-1987-4933-9c27-e98b5b15dc32" volumeName="kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665751    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="650a6586-f42f-4ba9-91b5-375372439c2b" volumeName="kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665770    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="650a6586-f42f-4ba9-91b5-375372439c2b" volumeName="kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665784    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665796    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69348615-8e95-479b-b360-6991eff92a2e" volumeName="kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb" seLinuxMountContext=""
Nov 01 03:07:13.665805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665809    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c2fda3f7-5f7d-401d-9bc7-42116112b337" volumeName="kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665822    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="650a6586-f42f-4ba9-91b5-375372439c2b" volumeName="kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665835    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665848    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8961e22c-8fdf-4994-9583-d2d1b5a7caa7" volumeName="kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665861    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8961e22c-8fdf-4994-9583-d2d1b5a7caa7" volumeName="kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665873    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8961e22c-8fdf-4994-9583-d2d1b5a7caa7" volumeName="kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665903    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b2e9266-2c96-49c3-8d2f-763aeb410cd2" volumeName="kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665914    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb94a967-7b3e-45b4-b117-0b591882dc95" volumeName="kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665926    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d4321cab-6240-420f-8920-aa78d1f91105" volumeName="kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665938    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26425cb6-d11f-4434-ba47-732c8fe23280" volumeName="kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665950    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="650a6586-f42f-4ba9-91b5-375372439c2b" volumeName="kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665963    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.665998    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666010    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="65f56877-d0bb-46b0-ab58-560c54fa29f8" volumeName="kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666020    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="69348615-8e95-479b-b360-6991eff92a2e" volumeName="kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666030    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b2e9266-2c96-49c3-8d2f-763aeb410cd2" volumeName="kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666040    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="da1ddb40-13f0-47ff-be1e-bbc395daf7f6" volumeName="kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666052    4892 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="26425cb6-d11f-4434-ba47-732c8fe23280" volumeName="kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned" seLinuxMountContext=""
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666063    4892 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 03:07:13.666309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.666072    4892 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 03:07:13.682306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.682281    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:07:13.682587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.682566    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:07:13.692108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.692078    4892 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 03:07:13.693305 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.693285    4892 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 03:07:13.693401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.693309    4892 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 03:07:13.693401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.693333    4892 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 03:07:13.693401 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.693374    4892 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 01 03:07:13.707547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707520    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="86610775e7c6cf24fa4706795e105a4927b80136dc5570a209568af5040dcf34" exitCode=0
Nov 01 03:07:13.707547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707544    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="a3e0e72e5c3414e8f5b7980c2d180bf95dd90659723ae6d12b151acbf4086810" exitCode=0
Nov 01 03:07:13.707547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707551    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="9f6e6e35127d5b6ac71c392a61697f724d40722770c1b2ba137f879f89f15479" exitCode=0
Nov 01 03:07:13.707739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707558    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="e27fcad73c876e4c83404a0240b62699a96bf943113574af59a5a96d2f2058b9" exitCode=0
Nov 01 03:07:13.707739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707563    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="79b96a001e3c9cee00c61a097fef9eccf2bf38fa8c67e2e205768c7de5fcb5e1" exitCode=0
Nov 01 03:07:13.707739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.707571    4892 generic.go:356] "Generic (PLEG): container finished" podID="18679b6e-26b2-4b4d-8169-d30c6e45d201" containerID="d63c61aa03251981a830f4517eb977d3f7ab8786ce28c1053a2e71ba37a3f3d0" exitCode=0
Nov 01 03:07:13.711364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.711344    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:07:13.711727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.711708    4892 generic.go:356] "Generic (PLEG): container finished" podID="69348615-8e95-479b-b360-6991eff92a2e" containerID="f46eb3a4b5b71bf310e0a8e7ca53143a3ae34eea271aad8a3e73b679e5cd8d6d" exitCode=1
Nov 01 03:07:13.737027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.737003    4892 generic.go:356] "Generic (PLEG): container finished" podID="df94e9eb892b28831be741642f228e6d" containerID="e1f6ae699c72c9ed199ddfd7d9b4d7f3f31b536f115d71b7575cee1d248b66ec" exitCode=0
Nov 01 03:07:13.740717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.740699    4892 manager.go:324] Recovery completed
Nov 01 03:07:13.756578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.756554    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.758661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.758645    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.758735 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.758675    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.758735 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.758685    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.761152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761135    4892 cpu_manager.go:226] "Starting" policy="none"
Nov 01 03:07:13.761152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761148    4892 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 03:07:13.761258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761158    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.761258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761166    4892 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 03:07:13.761321 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761306    4892 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 01 03:07:13.761365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761317    4892 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 01 03:07:13.761473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761462    4892 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 01 03:07:13.761504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761473    4892 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 01 03:07:13.761504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761479    4892 policy_none.go:50] "Start"
Nov 01 03:07:13.761504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761487    4892 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 03:07:13.761504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.761498    4892 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 03:07:13.761640 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.761513    4892 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:07:13.762250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762234    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.762333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762255    4892 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 01 03:07:13.762333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762263    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.762333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762268    4892 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 01 03:07:13.762333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762276    4892 policy_none.go:44] "Start"
Nov 01 03:07:13.762333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.762279    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.766788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.766762    4892 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 03:07:13.769414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769397    4892 manager.go:341] "Starting Device Plugin manager"
Nov 01 03:07:13.769494 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.769423    4892 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 03:07:13.769494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769431    4892 server.go:87] "Starting device plugin registration server"
Nov 01 03:07:13.769697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769684    4892 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 03:07:13.769756 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769695    4892 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 03:07:13.769803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769771    4892 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 03:07:13.769900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769864    4892 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 03:07:13.769965 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769910    4892 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 03:07:13.769965 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769917    4892 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 03:07:13.770090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.769970    4892 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T03:07:13.76989743Z","UUID":"6a09aa3e-3f15-4d07-a8a4-7636b0bc9c30","Handler":null,"Name":"","Endpoint":""}
Nov 01 03:07:13.771259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.771240    4892 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 03:07:13.771344 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.771277    4892 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 03:07:13.771344 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.771303    4892 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 03:07:13.771344 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.771335    4892 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-33.us-west-1.compute.internal\" not found"
Nov 01 03:07:13.793943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.793855    4892 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"]
Nov 01 03:07:13.794089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.794006    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.795054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.795040    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.795104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.795070    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.795104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.795083    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.795340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.795329    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.795494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.795472    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.796306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796291    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.796393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796309    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.796393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796318    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.796393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796330    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.796393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796334    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.796393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796341    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.796561 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.796482    4892 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.796600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796572    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"726f6b8e587032dd5c6b3f5ea9e72ff8","Type":"ContainerStarted","Data":"a111cd2dd6e26077d014d13a202f85adc4e7b24ca0369a56ae544924b7382a95"}
Nov 01 03:07:13.796631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796604    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"726f6b8e587032dd5c6b3f5ea9e72ff8","Type":"ContainerStarted","Data":"b831e8d2509fb749fafff8ffd79a93c6d53374a4c884c207c3e8f6fb2a8e8c2d"}
Nov 01 03:07:13.796631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796629    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.796715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796638    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerStarted","Data":"5a08f318f4bdaca2b5ea74bcdad5f7d45562ae08e7e44185ba7235273d5baff7"}
Nov 01 03:07:13.796715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796651    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerDied","Data":"e1f6ae699c72c9ed199ddfd7d9b4d7f3f31b536f115d71b7575cee1d248b66ec"}
Nov 01 03:07:13.796715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.796662    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal" event={"ID":"df94e9eb892b28831be741642f228e6d","Type":"ContainerStarted","Data":"2e933d8b10c110b38c15ea09499073cd716b776cbda45b969df5a0d4de40a314"}
Nov 01 03:07:13.797519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.797505    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.797591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.797528    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.797591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.797538    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.797667 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:13.797656    4892 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-33.us-west-1.compute.internal\" not found" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.860848 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.860809    4892 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:07:13.861036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.860915    4892 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 01 03:07:13.868713 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.868695    4892 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:07:13.868811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.868752    4892 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:07:13.870784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.870770    4892 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 03:07:13.871927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.871911    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientMemory"
Nov 01 03:07:13.872024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.871943    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 03:07:13.872024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.871956    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeHasSufficientPID"
Nov 01 03:07:13.872024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.872008    4892 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.919264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.919240    4892 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 03:07:13.961209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.961187    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.961209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.961215    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.961344 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.961238    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:13.962257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:13.962244    4892 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.010455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.010429    4892 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.010535 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.010525    4892 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.011114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.011097    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeNotReady"
Nov 01 03:07:14.011175 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.011133    4892 setters.go:546] "Node became not ready" node="ip-10-0-134-33.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-01T03:07:14Z","lastTransitionTime":"2026-11-01T03:07:14Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"}
Nov 01 03:07:14.043541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.043520    4892 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:07:14.043607 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:14.043567    4892 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.043607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.043579    4892 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061642    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061673    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/726f6b8e587032dd5c6b3f5ea9e72ff8-config\") pod \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"726f6b8e587032dd5c6b3f5ea9e72ff8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061683    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061702    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061729    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.061820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.061745    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/df94e9eb892b28831be741642f228e6d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" (UID: \"df94e9eb892b28831be741642f228e6d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.091669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.091640    4892 nodeinfomanager.go:471] existing CSINode "ip-10-0-134-33.us-west-1.compute.internal" is owned by different node (oldNodeID="8bf2202f-e305-4d3d-a6a6-8979ccdcc279", newNodeID=""), cleaning up...
Nov 01 03:07:14.124730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.124704    4892 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:07:14.124851 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:14.124756    4892 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.641346 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.641315    4892 apiserver.go:52] "Watching apiserver"
Nov 01 03:07:14.740051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.740025    4892 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.740199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.740158    4892 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.802602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.802571    4892 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 03:07:14.803063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.803040    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz","openshift-network-operator/iptables-alerter-frqbw","kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal","openshift-ingress-canary/ingress-canary-s4nk9","openshift-multus/multus-additional-cni-plugins-pw9lf","openshift-cluster-node-tuning-operator/tuned-pnjx6","openshift-image-registry/image-registry-5ff574547c-4g4gl","openshift-ovn-kubernetes/ovnkube-node-nrvct","openshift-console/downloads-6dd789c8b7-69f45","openshift-image-registry/node-ca-2lbxr","kube-system/konnectivity-agent-d72l5","openshift-dns/dns-default-79zbg","kube-system/global-pull-secret-syncer-kkdq4","openshift-insights/insights-runtime-extractor-9zmr7","openshift-multus/multus-n6qp4","openshift-multus/network-metrics-daemon-9r9sn","openshift-network-diagnostics/network-check-target-h2vwz","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f","openshift-dns/node-resolver-8ddz9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"]
Nov 01 03:07:14.804317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.804284    4892 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:07:14.804454 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:14.804359    4892 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.809179 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.809150    4892 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 03:07:14.809546 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:14.809522    4892 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-33.us-west-1.compute.internal"
Nov 01 03:07:14.863454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.863427    4892 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 03:07:14.864619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864600    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.864668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864625    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.864668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864646    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:07:14.864668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864661    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:07:14.864811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864729    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.864811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864763    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.864811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864781    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.864811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864799    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.864935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864821    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.864935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864859    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.864935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864878    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.864935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864908    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864954    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.864993    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865018    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865058    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-etc-tuned\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865063    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/650a6586-f42f-4ba9-91b5-375372439c2b-data-volume\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.865072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865072    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6827n\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.865262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865093    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.865262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865201    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.865262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865217    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.865262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865235    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.865379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865296    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:14.865379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865344    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.865447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865378    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.865447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865400    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.865447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865422    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.865557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865447    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.865557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865486    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.865557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865509    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:14.865557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865526    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.865557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865543    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.865715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865558    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.865715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865648    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-tmp\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:14.865715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865675    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.865715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865704    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.865832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865720    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.865832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865783    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.865832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865801    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4nnd\" (UniqueName: \"kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.865832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865817    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865853    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865879    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865896    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865903    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865919    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865941    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 03:07:14.865959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865944    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.865966    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866078    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866104    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866083    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866122    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866142    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbrgx\" (UniqueName: \"kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866159    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866175    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866189    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.866231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866225    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65f56877-d0bb-46b0-ab58-560c54fa29f8-ca-trust-extracted\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866246    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866280    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866306    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866333    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc5h9\" (UniqueName: \"kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866356    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866445    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.866500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866486    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.866683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866526    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.866683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866557    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.866683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866590    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfwbf\" (UniqueName: \"kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:14.866683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866617    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:14.866683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866633    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx6vt\" (UniqueName: \"kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.866832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866683    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.866832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866750    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.866832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866771    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.866832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866795    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.866939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866847    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.866939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866878    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.866939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866906    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.867035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.866955    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsrhz\" (UniqueName: \"kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.867035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867002    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.867125 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867108    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.867155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867140    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867168    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.867215 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867203    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.867274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867263    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-tmp-dir\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.867304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867269    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.867304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867297    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rlhw\" (UniqueName: \"kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.867358 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867323    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867356    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867372    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqpfb\" (UniqueName: \"kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867387    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.867404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867402    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.867531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867515    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/26425cb6-d11f-4434-ba47-732c8fe23280-tmp\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.867564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867512    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.867593 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867560    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.867621 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867594    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.867654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867615    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867634    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.867714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867651    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.867714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867680    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmqwm\" (UniqueName: \"kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.867770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867717    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttw5v\" (UniqueName: \"kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.867860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867847    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.867949 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867934    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxkth\" (UniqueName: \"kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.868004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.867995    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8pm4\" (UniqueName: \"kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:14.868073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868060    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 03:07:14.868148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868128    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.868201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868181    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.868261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868203    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.868261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868225    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.868261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868206    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.868261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868243    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-q29mz\" (UID: \"c2fda3f7-5f7d-401d-9bc7-42116112b337\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868260    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868276    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868285    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868288    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868310    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868337    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868367    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtn42\" (UniqueName: \"kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868373    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb94a967-7b3e-45b4-b117-0b591882dc95-tmp-dir\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868382    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868397    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868420    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868424    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.868447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868438    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.868807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868640    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2bc70ed0-1987-4933-9c27-e98b5b15dc32-iptables-alerter-script\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.868807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.868687    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 03:07:14.875790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.875765    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.875790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.875779    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-config\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.875921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.875826    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-certificates\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.877340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.877323    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-serviceca\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.878933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.878917    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-ovnkube-script-lib\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.880442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880421    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 03:07:14.880624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880608    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 03:07:14.880697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880624    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hrmfk\""
Nov 01 03:07:14.880697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880668    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-6592m\""
Nov 01 03:07:14.880697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880673    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mt5gq\""
Nov 01 03:07:14.880793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880673    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z9gxp\""
Nov 01 03:07:14.880793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880682    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-kbjpv\""
Nov 01 03:07:14.880793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880777    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 03:07:14.880872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880821    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-74fv2\""
Nov 01 03:07:14.880872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880860    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62njk\""
Nov 01 03:07:14.880950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880930    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ksxvj\""
Nov 01 03:07:14.881063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.880996    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 03:07:14.881063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.881000    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-wgrq6\""
Nov 01 03:07:14.881063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.881023    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-77n5m\""
Nov 01 03:07:14.881063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.881004    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4hc7l\""
Nov 01 03:07:14.885806 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.885789    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02652f44-f471-469d-bf6d-74236195ce03-metrics-certs\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:14.885877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.885829    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/43635a20-c47b-45c7-9616-8503d796561e-agent-certs\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:07:14.888070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.888054    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/69348615-8e95-479b-b360-6991eff92a2e-ovn-node-metrics-cert\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.904497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.904448    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 03:07:14.904497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.904457    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dg4zg\""
Nov 01 03:07:14.904497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.904486    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-vzcml\""
Nov 01 03:07:14.905704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.905688    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/650a6586-f42f-4ba9-91b5-375372439c2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.915598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.915581    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 01 03:07:14.915598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.915589    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-585dd\""
Nov 01 03:07:14.915683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.915612    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 03:07:14.916061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.916046    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-4lwfh\""
Nov 01 03:07:14.916123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.916067    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d4321cab-6240-420f-8920-aa78d1f91105-original-pull-secret\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.916188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.916174    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 03:07:14.916229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.916186    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 03:07:14.917432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.917418    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-cert\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:14.917887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.917872    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-fdfjz\""
Nov 01 03:07:14.918128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.918116    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 03:07:14.918188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.918133    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8s4ww\""
Nov 01 03:07:14.918188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.918159    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kkkrb\""
Nov 01 03:07:14.918188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.918162    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 03:07:14.918289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.918230    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 03:07:14.919525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.919509    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/c2fda3f7-5f7d-401d-9bc7-42116112b337-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-q29mz\" (UID: \"c2fda3f7-5f7d-401d-9bc7-42116112b337\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:14.926766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.926749    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-image-registry-private-configuration\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.927895 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.927878    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65f56877-d0bb-46b0-ab58-560c54fa29f8-installation-pull-secrets\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.928030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.928015    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-registry-tls\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.928065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.928030    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-metrics-tls\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.936459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.936436    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfwbf\" (UniqueName: \"kubernetes.io/projected/02652f44-f471-469d-bf6d-74236195ce03-kube-api-access-pfwbf\") pod \"network-metrics-daemon-9r9sn\" (UID: \"02652f44-f471-469d-bf6d-74236195ce03\") " pod="openshift-multus/network-metrics-daemon-9r9sn"
Nov 01 03:07:14.943622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.943606    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-bound-sa-token\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.959254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.959231    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmqwm\" (UniqueName: \"kubernetes.io/projected/18679b6e-26b2-4b4d-8169-d30c6e45d201-kube-api-access-pmqwm\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.959347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.959282    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqpfb\" (UniqueName: \"kubernetes.io/projected/69348615-8e95-479b-b360-6991eff92a2e-kube-api-access-rqpfb\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.960575 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.960554    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxkth\" (UniqueName: \"kubernetes.io/projected/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-kube-api-access-pxkth\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.967290 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967265    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.967373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967314    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 03:07:14.967373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967331    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.967373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967331    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 03:07:14.967373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967352    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 03:07:14.967373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967367    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.967569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967555    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.967615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967557    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.967615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967573    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 03:07:14.967615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967592    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.967721 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967624    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 03:07:14.967823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967802    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.967901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967831    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 03:07:14.967901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967843    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.967901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967865    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.968046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.967914    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.968046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968013    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.968366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968354    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.968397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968376    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 03:07:14.968685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968670    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 03:07:14.968739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968726    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.968775 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968750    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.968775 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968770    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.968845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968786    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.968845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968811    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968860    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968886    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968893    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968909    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968860    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-k8s-cni-cncf-io\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968920    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-cnibin\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.968943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968932    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-lib-modules\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968966    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2bc70ed0-1987-4933-9c27-e98b5b15dc32-host-slash\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969009    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-conf\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969035    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx6vt\" (UniqueName: \"kubernetes.io/projected/650a6586-f42f-4ba9-91b5-375372439c2b-kube-api-access-xx6vt\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969022    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-systemd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.968929    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969060    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-kubelet-config\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969042    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-ovn\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969083    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969109    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-log-socket\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969113    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969156    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969159    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-etc-selinux\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969182    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969199    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969209    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969203    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969240    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-host\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.969263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969262    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-sys\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969292    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969311    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-netns\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969321    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969349    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969374    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969401    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969406    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-kubelet\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969428    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969437    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cb94a967-7b3e-45b4-b117-0b591882dc95-hosts-file\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969452    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-var-lib-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969453    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969461    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-system-cni-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969485    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-device-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969505    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rlhw\" (UniqueName: \"kubernetes.io/projected/cb94a967-7b3e-45b4-b117-0b591882dc95-kube-api-access-2rlhw\") pod \"node-resolver-8ddz9\" (UID: \"cb94a967-7b3e-45b4-b117-0b591882dc95\") " pod="openshift-dns/node-resolver-8ddz9"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969534    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969542    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-kubernetes\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969553    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969575    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-hostroot\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969578    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-registration-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969591    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969622    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-sys-fs\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969629    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969634    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzf2x\" (UniqueName: \"kubernetes.io/projected/d440d7f0-e153-48ea-94fd-2e5bb181c0a7-kube-api-access-zzf2x\") pod \"network-check-target-h2vwz\" (UID: \"d440d7f0-e153-48ea-94fd-2e5bb181c0a7\") " pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969653    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969661    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-socket-dir-parent\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969678    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969697    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969711    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969739    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/18679b6e-26b2-4b4d-8169-d30c6e45d201-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969773    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-systemd\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969776    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-run-multus-certs\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969777    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d4321cab-6240-420f-8920-aa78d1f91105-dbus\") pod \"global-pull-secret-syncer-kkdq4\" (UID: \"d4321cab-6240-420f-8920-aa78d1f91105\") " pod="kube-system/global-pull-secret-syncer-kkdq4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969781    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969819    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-multus\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969837    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969860    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-etc-kubernetes\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.969853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969864    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969885    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969887    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-bin\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969843    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969903    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969921    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969935    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-host\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969939    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969957    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969961    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-conf-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.969992    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970006    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-systemd-units\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970015    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970021    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/00459c43-88b4-494f-a76f-3cbef932a916-socket-dir\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970033    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-netns\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970050    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970056    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-cni-netd\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970070    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970081    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-modprobe-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970094    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970114    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970121    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-run-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970129    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970093    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-host-var-lib-cni-bin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970131    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970157    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-kubelet\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970172    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-system-cni-dir\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970174    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970209    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-host-slash\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970219    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970241    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970256    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970270    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970292    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970323    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-run\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970339    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-run-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970341    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/650a6586-f42f-4ba9-91b5-375372439c2b-crio-socket\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970354    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsrhz\" (UniqueName: \"kubernetes.io/projected/26425cb6-d11f-4434-ba47-732c8fe23280-kube-api-access-fsrhz\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970359    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970361    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysconfig\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970377    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-etc-sysctl-d\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970389    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970413    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970419    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26425cb6-d11f-4434-ba47-732c8fe23280-var-lib-kubelet\") pod \"tuned-pnjx6\" (UID: \"26425cb6-d11f-4434-ba47-732c8fe23280\") " pod="openshift-cluster-node-tuning-operator/tuned-pnjx6"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970429    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970438    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cnibin\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970457    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-etc-openvswitch\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.971035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.970471    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/69348615-8e95-479b-b360-6991eff92a2e-node-log\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.972753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.971595    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtn42\" (UniqueName: \"kubernetes.io/projected/0d865cd5-54e7-4607-b0a4-a54ca52f61cd-kube-api-access-rtn42\") pod \"downloads-6dd789c8b7-69f45\" (UID: \"0d865cd5-54e7-4607-b0a4-a54ca52f61cd\") " pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:14.972753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.972287    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4nnd\" (UniqueName: \"kubernetes.io/projected/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-kube-api-access-j4nnd\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.972753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.972427    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttw5v\" (UniqueName: \"kubernetes.io/projected/00459c43-88b4-494f-a76f-3cbef932a916-kube-api-access-ttw5v\") pod \"aws-ebs-csi-driver-node-fwq4f\" (UID: \"00459c43-88b4-494f-a76f-3cbef932a916\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fwq4f"
Nov 01 03:07:14.974595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.974573    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6827n\" (UniqueName: \"kubernetes.io/projected/65f56877-d0bb-46b0-ab58-560c54fa29f8-kube-api-access-6827n\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.975423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.975407    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/43635a20-c47b-45c7-9616-8503d796561e-konnectivity-ca\") pod \"konnectivity-agent-d72l5\" (UID: \"43635a20-c47b-45c7-9616-8503d796561e\") " pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:07:14.975423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.975414    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-cni-binary-copy\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.975663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.975648    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc5h9\" (UniqueName: \"kubernetes.io/projected/1a4e4f55-5014-48d0-98cc-669a1cce1dd8-kube-api-access-wc5h9\") pod \"node-ca-2lbxr\" (UID: \"1a4e4f55-5014-48d0-98cc-669a1cce1dd8\") " pod="openshift-image-registry/node-ca-2lbxr"
Nov 01 03:07:14.976622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.976604    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/69348615-8e95-479b-b360-6991eff92a2e-env-overrides\") pod \"ovnkube-node-nrvct\" (UID: \"69348615-8e95-479b-b360-6991eff92a2e\") " pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:14.976676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.976627    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-cni-binary-copy\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.976676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.976632    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/18679b6e-26b2-4b4d-8169-d30c6e45d201-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pw9lf\" (UID: \"18679b6e-26b2-4b4d-8169-d30c6e45d201\") " pod="openshift-multus/multus-additional-cni-plugins-pw9lf"
Nov 01 03:07:14.976916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.976902    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbrgx\" (UniqueName: \"kubernetes.io/projected/2bc70ed0-1987-4933-9c27-e98b5b15dc32-kube-api-access-wbrgx\") pod \"iptables-alerter-frqbw\" (UID: \"2bc70ed0-1987-4933-9c27-e98b5b15dc32\") " pod="openshift-network-operator/iptables-alerter-frqbw"
Nov 01 03:07:14.977102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.977085    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8961e22c-8fdf-4994-9583-d2d1b5a7caa7-config-volume\") pod \"dns-default-79zbg\" (UID: \"8961e22c-8fdf-4994-9583-d2d1b5a7caa7\") " pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:14.977141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.977112    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/650a6586-f42f-4ba9-91b5-375372439c2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9zmr7\" (UID: \"650a6586-f42f-4ba9-91b5-375372439c2b\") " pod="openshift-insights/insights-runtime-extractor-9zmr7"
Nov 01 03:07:14.978909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.978895    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b2e9266-2c96-49c3-8d2f-763aeb410cd2-multus-daemon-config\") pod \"multus-n6qp4\" (UID: \"8b2e9266-2c96-49c3-8d2f-763aeb410cd2\") " pod="openshift-multus/multus-n6qp4"
Nov 01 03:07:14.979026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.979011    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65f56877-d0bb-46b0-ab58-560c54fa29f8-trusted-ca\") pod \"image-registry-5ff574547c-4g4gl\" (UID: \"65f56877-d0bb-46b0-ab58-560c54fa29f8\") " pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:14.979478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:14.979464    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8pm4\" (UniqueName: \"kubernetes.io/projected/da1ddb40-13f0-47ff-be1e-bbc395daf7f6-kube-api-access-m8pm4\") pod \"ingress-canary-s4nk9\" (UID: \"da1ddb40-13f0-47ff-be1e-bbc395daf7f6\") " pod="openshift-ingress-canary/ingress-canary-s4nk9"
Nov 01 03:07:15.113000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.112967    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:07:15.676750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.676715    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-psfb8"]
Nov 01 03:07:15.679676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.679655    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.758395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.758371    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 01 03:07:15.758549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.758519    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 01 03:07:15.758549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.758533    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 01 03:07:15.758799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.758787    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 01 03:07:15.764961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.764939    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:15.776850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.776822    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.776960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.776858    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv5bw\" (UniqueName: \"kubernetes.io/projected/3b8ac128-4c61-4af8-96b7-fd384ae50e07-kube-api-access-fv5bw\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.776960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.776882    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-metrics-client-ca\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777078 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777000    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-wtmp\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777153 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777137    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-tls\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777173    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-root\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777276    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-textfile\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777343 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777306    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-accelerators-collector-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.777435 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.777416    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-sys\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.824659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.824631    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 01 03:07:15.824904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.824886    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 01 03:07:15.824904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.824898    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jfdhs\""
Nov 01 03:07:15.878222 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878188    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-textfile\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878228    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-accelerators-collector-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878251    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-sys\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878305    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-sys\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878368    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878418    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv5bw\" (UniqueName: \"kubernetes.io/projected/3b8ac128-4c61-4af8-96b7-fd384ae50e07-kube-api-access-fv5bw\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878444    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-metrics-client-ca\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878484    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-wtmp\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878511    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-tls\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878536    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-root\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878624    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-textfile\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878631    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-root\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878664    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-wtmp\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.878818 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878741    4892 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 03:07:15.878903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.878885    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-accelerators-collector-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.879255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.879239    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b8ac128-4c61-4af8-96b7-fd384ae50e07-metrics-client-ca\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.880798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.880776    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.880889 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.880843    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b8ac128-4c61-4af8-96b7-fd384ae50e07-node-exporter-tls\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.943054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.942973    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv5bw\" (UniqueName: \"kubernetes.io/projected/3b8ac128-4c61-4af8-96b7-fd384ae50e07-kube-api-access-fv5bw\") pod \"node-exporter-psfb8\" (UID: \"3b8ac128-4c61-4af8-96b7-fd384ae50e07\") " pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:15.992226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:15.992192    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-psfb8"
Nov 01 03:07:16.001345 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:16.001318    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8ac128_4c61_4af8_96b7_fd384ae50e07.slice/crio-44b4709c3351ac0bb09bab6da7f839df76f3c16a3d491b695170c07dc7b87f2b WatchSource:0}: Error finding container 44b4709c3351ac0bb09bab6da7f839df76f3c16a3d491b695170c07dc7b87f2b: Status 404 returned error can't find the container with id 44b4709c3351ac0bb09bab6da7f839df76f3c16a3d491b695170c07dc7b87f2b
Nov 01 03:07:16.749453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:16.749418    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psfb8" event={"ID":"3b8ac128-4c61-4af8-96b7-fd384ae50e07","Type":"ContainerStarted","Data":"44b4709c3351ac0bb09bab6da7f839df76f3c16a3d491b695170c07dc7b87f2b"}
Nov 01 03:07:17.306570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:17.306533    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:17.311199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:17.311170    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-q29mz"
Nov 01 03:07:17.754685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:17.754648    4892 generic.go:356] "Generic (PLEG): container finished" podID="3b8ac128-4c61-4af8-96b7-fd384ae50e07" containerID="92a3448bf4bf6c65001868c4986a870daef6918634167b567c758f6e5d2733b3" exitCode=0
Nov 01 03:07:17.755138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:17.754729    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psfb8" event={"ID":"3b8ac128-4c61-4af8-96b7-fd384ae50e07","Type":"ContainerDied","Data":"92a3448bf4bf6c65001868c4986a870daef6918634167b567c758f6e5d2733b3"}
Nov 01 03:07:18.766466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:18.766416    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psfb8" event={"ID":"3b8ac128-4c61-4af8-96b7-fd384ae50e07","Type":"ContainerStarted","Data":"1749c1d2b71520d170aa37b9346e79edebb43f541b229fec3997ac342acadac1"}
Nov 01 03:07:18.766920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:18.766514    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-psfb8" event={"ID":"3b8ac128-4c61-4af8-96b7-fd384ae50e07","Type":"ContainerStarted","Data":"9449a9a06f5b3720caa9926d0de57eff200fe2cfc3de4d4125ae792159685afd"}
Nov 01 03:07:18.767454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:18.767421    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-79zbg"
Nov 01 03:07:18.891199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:18.891142    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-psfb8" podStartSLOduration=2.404232711 podStartE2EDuration="3.891130018s" totalImagesPullingTime="1.402001842s" totalInitContainerRuntime="84.895465ms" isStatefulPod=true podCreationTimestamp="2026-11-01 03:07:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:18.89018173 +0000 UTC m=+5.306081262" watchObservedRunningTime="2026-11-01 03:07:18.891130018 +0000 UTC m=+5.307029547"
Nov 01 03:07:19.031219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:19.031142    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:20.001547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:20.001520    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:20.002161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:20.001792    4892 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 03:07:20.129994 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:20.129957    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nrvct"
Nov 01 03:07:22.090921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:22.090829    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:22.095685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:22.095659    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5ff574547c-4g4gl"
Nov 01 03:07:24.220945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.220916    4892 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-33.us-west-1.compute.internal" event="NodeReady"
Nov 01 03:07:24.372728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.372688    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:24.936602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.936570    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:07:24.941169 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.941139    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:24.997996 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.997928    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 01 03:07:24.998133 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.997995    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 01 03:07:24.998177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998149    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 01 03:07:24.998177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998163    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tnv44\""
Nov 01 03:07:24.998490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998460    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 01 03:07:24.998631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998506    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 01 03:07:24.998762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998735    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 01 03:07:24.998851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:24.998819    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 01 03:07:25.048226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.048181    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 01 03:07:25.048226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.048186    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 01 03:07:25.048823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.048769    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 01 03:07:25.049379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.049294    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 01 03:07:25.055292 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055265    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055313    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055344    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055369    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fm6q\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055481    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055506    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055528    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055542    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055565    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055588    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055605    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055622    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055638    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055660    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055705    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.055779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055770    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.056356 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055852    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.056356 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.055972    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.056356 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.056038    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.056356 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.056137    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.148923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.148878    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 01 03:07:25.148923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.148889    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 01 03:07:25.149180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.149042    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 01 03:07:25.149180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.149061    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c3svv7dqk8fft\""
Nov 01 03:07:25.149180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.149108    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 01 03:07:25.149940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.149922    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:07:25.156856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156833    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.156960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156861    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.156960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156887    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.156960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156910    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.156960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156934    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fm6q\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.156963    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157009    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157040    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157065    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157103    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157130    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157156    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157182    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157209    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157235    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157259    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157280    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157299    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157333    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157355    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.157851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.157711    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.159573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.158580    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.159573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.159231    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.160634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.160604    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.160909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.160843    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.161012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.160903    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.161254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.161225    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162050    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162465    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162472    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162692    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162781 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162743    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.162951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.162928    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.163171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.163129    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.163242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.163185    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.163553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.163531    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.164045 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.164020    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.164921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.164898    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.165164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.165145    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.225762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.225693    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fm6q\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\") pod \"prometheus-k8s-0\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.270709 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.270672    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:25.722549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.722519    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:07:25.725945 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:07:25.725918    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afc90fa_180b_4d0b_9d93_fb742ddc10f8.slice/crio-f0140241eafb81a92855133b27dc92926c29943173c35192693b552e4e19e83d WatchSource:0}: Error finding container f0140241eafb81a92855133b27dc92926c29943173c35192693b552e4e19e83d: Status 404 returned error can't find the container with id f0140241eafb81a92855133b27dc92926c29943173c35192693b552e4e19e83d
Nov 01 03:07:25.792483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:25.792444    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"f0140241eafb81a92855133b27dc92926c29943173c35192693b552e4e19e83d"}
Nov 01 03:07:27.612181 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:27.612140    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:07:27.802086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:27.802051    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="88dcbd4cd3d7fa99a64fa3e85de166091acf28cecf0b72b5770ebf2a03a6e7a7" exitCode=0
Nov 01 03:07:27.802258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:27.802114    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"88dcbd4cd3d7fa99a64fa3e85de166091acf28cecf0b72b5770ebf2a03a6e7a7"}
Nov 01 03:07:29.752420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:29.752387    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-d72l5"
Nov 01 03:07:32.823788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:32.823745    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"e45526f03b00b044b8160f439fba3c6976b393465319af3e19d350b8d747dc0d"}
Nov 01 03:07:32.824293 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:32.823868    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"4400ee36612aabc22f97e1615b4a54f5e35c63472daa45810620e52702fcad74"}
Nov 01 03:07:32.825569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:32.825537    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-69f45" event={"ID":"0d865cd5-54e7-4607-b0a4-a54ca52f61cd","Type":"ContainerStarted","Data":"1864cee247e9e7e6f85ca05e3dfbf3914132b73b9fc237a3c5a8e319d7e43ff8"}
Nov 01 03:07:32.825872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:32.825822    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:32.846921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:32.846894    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-69f45"
Nov 01 03:07:35.841069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:35.840961    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"74146a088f901b8cb4d6dcb337dbb4a7514a54f2a3d9ec734e52e3032a30cc11"}
Nov 01 03:07:35.841069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:35.841063    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"dd587e93e19f8b1b6abd84665cd5a413d8a761b50c7e140216949d18e5cd280a"}
Nov 01 03:07:35.841584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:35.841108    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"986e2cc52fe5419870b4abac39d52f6d98d294aed9b67cadbd8be3c5f004db0d"}
Nov 01 03:07:35.841584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:35.841150    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerStarted","Data":"dbab8afcc4e09ab12b1076e152e50b40de8ef3ef68cac15db93fc11a57d2cc73"}
Nov 01 03:07:36.123869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:36.123757    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.453221248 podStartE2EDuration="12.123737898s" totalImagesPullingTime="8.64938186s" totalInitContainerRuntime="21.13479ms" isStatefulPod=false podCreationTimestamp="2026-11-01 03:07:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:07:36.073106704 +0000 UTC m=+22.489006232" watchObservedRunningTime="2026-11-01 03:07:36.123737898 +0000 UTC m=+22.539637429"
Nov 01 03:07:40.271645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:40.271602    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:07:44.376391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:07:44.376361    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-h2vwz"
Nov 01 03:07:57.608787 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:07:57.608752    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:08:27.609564 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:08:27.609525    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:08:31.442962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:31.442929    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:31.460742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:31.460716    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:32.036070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:32.036041    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:47.946843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.946801    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:47.947335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947296    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="prometheus" containerID="cri-o://4400ee36612aabc22f97e1615b4a54f5e35c63472daa45810620e52702fcad74" gracePeriod=600
Nov 01 03:08:47.947473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947348    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="thanos-sidecar" containerID="cri-o://dbab8afcc4e09ab12b1076e152e50b40de8ef3ef68cac15db93fc11a57d2cc73" gracePeriod=600
Nov 01 03:08:47.947473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947348    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-web" containerID="cri-o://986e2cc52fe5419870b4abac39d52f6d98d294aed9b67cadbd8be3c5f004db0d" gracePeriod=600
Nov 01 03:08:47.947473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947378    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://74146a088f901b8cb4d6dcb337dbb4a7514a54f2a3d9ec734e52e3032a30cc11" gracePeriod=600
Nov 01 03:08:47.947473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947379    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="config-reloader" containerID="cri-o://e45526f03b00b044b8160f439fba3c6976b393465319af3e19d350b8d747dc0d" gracePeriod=600
Nov 01 03:08:47.947473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:47.947429    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy" containerID="cri-o://dd587e93e19f8b1b6abd84665cd5a413d8a761b50c7e140216949d18e5cd280a" gracePeriod=600
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077385    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="74146a088f901b8cb4d6dcb337dbb4a7514a54f2a3d9ec734e52e3032a30cc11" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077417    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="dd587e93e19f8b1b6abd84665cd5a413d8a761b50c7e140216949d18e5cd280a" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077430    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="986e2cc52fe5419870b4abac39d52f6d98d294aed9b67cadbd8be3c5f004db0d" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077441    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="dbab8afcc4e09ab12b1076e152e50b40de8ef3ef68cac15db93fc11a57d2cc73" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077465    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="e45526f03b00b044b8160f439fba3c6976b393465319af3e19d350b8d747dc0d" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077475    4892 generic.go:356] "Generic (PLEG): container finished" podID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerID="4400ee36612aabc22f97e1615b4a54f5e35c63472daa45810620e52702fcad74" exitCode=0
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077470    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"74146a088f901b8cb4d6dcb337dbb4a7514a54f2a3d9ec734e52e3032a30cc11"}
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077620    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"dd587e93e19f8b1b6abd84665cd5a413d8a761b50c7e140216949d18e5cd280a"}
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077674    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"986e2cc52fe5419870b4abac39d52f6d98d294aed9b67cadbd8be3c5f004db0d"}
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077727    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"dbab8afcc4e09ab12b1076e152e50b40de8ef3ef68cac15db93fc11a57d2cc73"}
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077769    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"e45526f03b00b044b8160f439fba3c6976b393465319af3e19d350b8d747dc0d"}
Nov 01 03:08:48.077855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.077805    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"4400ee36612aabc22f97e1615b4a54f5e35c63472daa45810620e52702fcad74"}
Nov 01 03:08:48.201239 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.201171    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:48.297868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297831    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.297868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297875    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297895    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297914    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297933    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.297958    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298006    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298045    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298078    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298116    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298160    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298190    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298223    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298275    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298310    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298325    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298355    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298367    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298401    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298442    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298473    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.298550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298506    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\") pod \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\" (UID: \"8afc90fa-180b-4d0b-9d93-fb742ddc10f8\") "
Nov 01 03:08:48.299173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298726    4892 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.299173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.298746    4892 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-metrics-client-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.299277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.299204    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.299998    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.300439    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.300535    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.300960    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.301023    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.301170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.301133    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:08:48.301553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.301326    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "kube-api-access-8fm6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:08:48.301553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.301515    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.301775 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.301749    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:08:48.302157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.302132    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.302721 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.302694    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:08:48.302838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.302811    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.303104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.303086    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.303274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.303258    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.303340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.303331    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.304010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.303969    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.311166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.311138    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config" pod "8afc90fa-180b-4d0b-9d93-fb742ddc10f8" (UID: "8afc90fa-180b-4d0b-9d93-fb742ddc10f8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:08:48.399689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399653    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fm6q\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-kube-api-access-8fm6q\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399684    4892 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399694    4892 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config-out\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399704    4892 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399714    4892 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399723    4892 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-config\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399732    4892 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-grpc-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399741    4892 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399749    4892 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399758    4892 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-db\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399767    4892 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399777    4892 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-metrics-client-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399786    4892 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399795    4892 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-tls-assets\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399805    4892 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-web-config\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399814    4892 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-kube-rbac-proxy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399822    4892 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:48.399905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:48.399845    4892 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8afc90fa-180b-4d0b-9d93-fb742ddc10f8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:08:49.084138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.084101    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8afc90fa-180b-4d0b-9d93-fb742ddc10f8","Type":"ContainerDied","Data":"f0140241eafb81a92855133b27dc92926c29943173c35192693b552e4e19e83d"}
Nov 01 03:08:49.084552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.084170    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.084552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.084207    4892 scope.go:122] "RemoveContainer" containerID="74146a088f901b8cb4d6dcb337dbb4a7514a54f2a3d9ec734e52e3032a30cc11"
Nov 01 03:08:49.093013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.092973    4892 scope.go:122] "RemoveContainer" containerID="dd587e93e19f8b1b6abd84665cd5a413d8a761b50c7e140216949d18e5cd280a"
Nov 01 03:08:49.100008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.099964    4892 scope.go:122] "RemoveContainer" containerID="986e2cc52fe5419870b4abac39d52f6d98d294aed9b67cadbd8be3c5f004db0d"
Nov 01 03:08:49.107229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.107206    4892 scope.go:122] "RemoveContainer" containerID="dbab8afcc4e09ab12b1076e152e50b40de8ef3ef68cac15db93fc11a57d2cc73"
Nov 01 03:08:49.113948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.113924    4892 scope.go:122] "RemoveContainer" containerID="e45526f03b00b044b8160f439fba3c6976b393465319af3e19d350b8d747dc0d"
Nov 01 03:08:49.121094 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.121074    4892 scope.go:122] "RemoveContainer" containerID="4400ee36612aabc22f97e1615b4a54f5e35c63472daa45810620e52702fcad74"
Nov 01 03:08:49.128465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.128443    4892 scope.go:122] "RemoveContainer" containerID="88dcbd4cd3d7fa99a64fa3e85de166091acf28cecf0b72b5770ebf2a03a6e7a7"
Nov 01 03:08:49.252935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.252905    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:49.302795 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.302724    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:49.303188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303166    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:49.303452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303433    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="init-config-reloader"
Nov 01 03:08:49.303452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303452    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="init-config-reloader"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303467    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="prometheus"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303472    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="prometheus"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303478    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="config-reloader"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303483    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="config-reloader"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303489    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-web"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303493    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-web"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303508    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303515    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303529    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="thanos-sidecar"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303534    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="thanos-sidecar"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303540    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-thanos"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303545    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-thanos"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303587    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy"
Nov 01 03:08:49.303595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303599    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="thanos-sidecar"
Nov 01 03:08:49.304087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303609    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="config-reloader"
Nov 01 03:08:49.304087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303616    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="prometheus"
Nov 01 03:08:49.304087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303623    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-web"
Nov 01 03:08:49.304087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.303629    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" containerName="kube-rbac-proxy-thanos"
Nov 01 03:08:49.307090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.307071    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.360250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.360218    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Nov 01 03:08:49.360250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.360225    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Nov 01 03:08:49.360519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.360344    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Nov 01 03:08:49.360739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.360718    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Nov 01 03:08:49.360969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.360939    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Nov 01 03:08:49.361099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361008    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c3svv7dqk8fft\""
Nov 01 03:08:49.361222 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361201    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Nov 01 03:08:49.361288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361269    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Nov 01 03:08:49.361349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361322    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Nov 01 03:08:49.361349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361323    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Nov 01 03:08:49.361349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361330    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Nov 01 03:08:49.361478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361363    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Nov 01 03:08:49.361478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.361367    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Nov 01 03:08:49.407168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407133    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407172    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407189    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407237    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407259    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407333    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407400    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407418    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407447    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407462    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407507    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407533    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407638 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407611    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407638 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407631    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407645    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407658    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407682    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407720    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407766    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.407854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.407801    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56vkz\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-kube-api-access-56vkz\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.410443 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.410427    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Nov 01 03:08:49.410943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.410926    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tnv44\""
Nov 01 03:08:49.411295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.411281    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Nov 01 03:08:49.411438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.411424    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Nov 01 03:08:49.425303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.425275    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:49.508390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508301    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508337    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508356    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508383    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508401    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508616    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508659    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508698    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508728    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508752    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508775    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508811    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508840    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508873    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.508927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508896    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56vkz\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-kube-api-access-56vkz\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.509299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508932    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.509299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.508958    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.509299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509005    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.509299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509043    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.509299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509073    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509402    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509855    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509883    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.509899    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.510403    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.511298    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.511957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.511634    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.512974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.512648    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7b78df2f-f10f-457f-acd4-206186a56bf7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.512974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.512896    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.513276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.513128    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.513701    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.513775    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.513932    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.514001    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.514044    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.514087    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7b78df2f-f10f-457f-acd4-206186a56bf7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.514259    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.514483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.514283    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.515571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.515544    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7b78df2f-f10f-457f-acd4-206186a56bf7-config\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.571369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.571331    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56vkz\" (UniqueName: \"kubernetes.io/projected/7b78df2f-f10f-457f-acd4-206186a56bf7-kube-api-access-56vkz\") pod \"prometheus-k8s-0\" (UID: \"7b78df2f-f10f-457f-acd4-206186a56bf7\") " pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.623876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.623834    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:49.700552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:49.700513    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8afc90fa-180b-4d0b-9d93-fb742ddc10f8" path="/var/lib/kubelet/pods/8afc90fa-180b-4d0b-9d93-fb742ddc10f8/volumes"
Nov 01 03:08:50.060137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:50.059948    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Nov 01 03:08:50.062969 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:08:50.062941    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b78df2f_f10f_457f_acd4_206186a56bf7.slice/crio-be08e81e9e79a2ddeb0cc51248023a25760dba14378b07c22934e1a9a393472d WatchSource:0}: Error finding container be08e81e9e79a2ddeb0cc51248023a25760dba14378b07c22934e1a9a393472d: Status 404 returned error can't find the container with id be08e81e9e79a2ddeb0cc51248023a25760dba14378b07c22934e1a9a393472d
Nov 01 03:08:50.090334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:50.090291    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"be08e81e9e79a2ddeb0cc51248023a25760dba14378b07c22934e1a9a393472d"}
Nov 01 03:08:51.095533 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:51.095487    4892 generic.go:356] "Generic (PLEG): container finished" podID="7b78df2f-f10f-457f-acd4-206186a56bf7" containerID="7a909597da8de65c2a0448e40658a0a5a0656e78f4d192595b41acc5b5ab3e77" exitCode=0
Nov 01 03:08:51.095935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:51.095570    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerDied","Data":"7a909597da8de65c2a0448e40658a0a5a0656e78f4d192595b41acc5b5ab3e77"}
Nov 01 03:08:52.106870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.106786    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"dd4f9da92c368b1a1dce1a6f5a1f6c6a4d1d2ee1d91f8e31875c6105bc0feb17"}
Nov 01 03:08:52.107255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.106904    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"c6545a75cfc4632d2cfb22a45c5d12a2b957d67bbff851ab4b3c0293dec658d7"}
Nov 01 03:08:52.107255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.106947    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"f5291313fc5582d95bf31fe968297c5273cd8cdf1d61bbcc01dd06d062b23239"}
Nov 01 03:08:52.107255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.106998    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"c678d3b99060c51fea7088952f2ee94165417f972110f2e20d20bb30787639bc"}
Nov 01 03:08:52.107255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.107036    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"f7589774516124263bb22072dca7b747771715d275d78d79f9a72e7630f5e112"}
Nov 01 03:08:52.107255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.107074    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7b78df2f-f10f-457f-acd4-206186a56bf7","Type":"ContainerStarted","Data":"692159f1ebe4db314274896c791522b3d722400c06c3c18b4facfeabf087306f"}
Nov 01 03:08:52.241248 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:52.241188    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.224848416 podStartE2EDuration="3.241173284s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.324868ms" isStatefulPod=false podCreationTimestamp="2026-11-01 03:08:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:08:52.240885663 +0000 UTC m=+98.656785231" watchObservedRunningTime="2026-11-01 03:08:52.241173284 +0000 UTC m=+98.657072813"
Nov 01 03:08:54.624469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:08:54.624422    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:08:57.610366 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:08:57.610314    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:09:27.610316 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:09:27.610278    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:09:49.624912 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:09:49.624868    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:09:49.640213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:09:49.640175    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:09:50.309432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:09:50.309391    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Nov 01 03:09:57.609422 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:09:57.609379    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:10:27.609652 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:10:27.609565    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:10:36.915587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:36.915549    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"]
Nov 01 03:10:36.917710 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:36.917692    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.020664 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.020630    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.020828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.020746    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.020828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.020775    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6nfr\" (UniqueName: \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.021417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.021398    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Nov 01 03:10:37.021483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.021438    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Nov 01 03:10:37.021529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.021508    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-47bcn\""
Nov 01 03:10:37.022195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.022177    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"]
Nov 01 03:10:37.122012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.121960    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.122182 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.122044    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.122182 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.122067    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6nfr\" (UniqueName: \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.122431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.122411    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.122468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.122438    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.184966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.184891    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6nfr\" (UniqueName: \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.229396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.229363    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:37.605663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:37.605633    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"]
Nov 01 03:10:37.609640 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:10:37.609609    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddef0ecae_b9e1_44ba_adc0_5070d623eb4f.slice/crio-d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd WatchSource:0}: Error finding container d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd: Status 404 returned error can't find the container with id d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd
Nov 01 03:10:38.448969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:38.448934    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw" event={"ID":"def0ecae-b9e1-44ba-adc0-5070d623eb4f","Type":"ContainerStarted","Data":"d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd"}
Nov 01 03:10:44.472468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:44.472439    4892 generic.go:356] "Generic (PLEG): container finished" podID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerID="f915e81f5df301a12824813e591ecee46e6268076ceb7ad3f9cc455e9ee0e5ac" exitCode=0
Nov 01 03:10:44.472839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:44.472503    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw" event={"ID":"def0ecae-b9e1-44ba-adc0-5070d623eb4f","Type":"ContainerDied","Data":"f915e81f5df301a12824813e591ecee46e6268076ceb7ad3f9cc455e9ee0e5ac"}
Nov 01 03:10:46.479806 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:46.479767    4892 generic.go:356] "Generic (PLEG): container finished" podID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerID="892dddfd782bbc312b660f754dfc820ed2c388c8de24de8c91ec2a2eef563f53" exitCode=0
Nov 01 03:10:46.480212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:46.479861    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw" event={"ID":"def0ecae-b9e1-44ba-adc0-5070d623eb4f","Type":"ContainerDied","Data":"892dddfd782bbc312b660f754dfc820ed2c388c8de24de8c91ec2a2eef563f53"}
Nov 01 03:10:55.514559 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:55.514525    4892 generic.go:356] "Generic (PLEG): container finished" podID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerID="1223ce5d91a9c3cecf4b003e2cc2b37fb80aaca5f621acecfa6628baebbdb23f" exitCode=0
Nov 01 03:10:55.514972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:55.514570    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw" event={"ID":"def0ecae-b9e1-44ba-adc0-5070d623eb4f","Type":"ContainerDied","Data":"1223ce5d91a9c3cecf4b003e2cc2b37fb80aaca5f621acecfa6628baebbdb23f"}
Nov 01 03:10:56.638093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.638068    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:56.781062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.780968    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\") pod \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") "
Nov 01 03:10:56.781062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.781033    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\") pod \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") "
Nov 01 03:10:56.781240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.781063    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\" (UniqueName: \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\") pod \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\" (UID: \"def0ecae-b9e1-44ba-adc0-5070d623eb4f\") "
Nov 01 03:10:56.781665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.781631    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle" pod "def0ecae-b9e1-44ba-adc0-5070d623eb4f" (UID: "def0ecae-b9e1-44ba-adc0-5070d623eb4f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:10:56.783455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.783425    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr" pod "def0ecae-b9e1-44ba-adc0-5070d623eb4f" (UID: "def0ecae-b9e1-44ba-adc0-5070d623eb4f"). InnerVolumeSpecName "kube-api-access-t6nfr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:10:56.785055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.785032    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util" pod "def0ecae-b9e1-44ba-adc0-5070d623eb4f" (UID: "def0ecae-b9e1-44ba-adc0-5070d623eb4f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:10:56.881792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.881756    4892 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-bundle\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:10:56.881792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.881785    4892 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/def0ecae-b9e1-44ba-adc0-5070d623eb4f-util\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:10:56.881792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:56.881795    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6nfr\" (UniqueName: \"kubernetes.io/projected/def0ecae-b9e1-44ba-adc0-5070d623eb4f-kube-api-access-t6nfr\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:10:57.522857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:57.522821    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw" event={"ID":"def0ecae-b9e1-44ba-adc0-5070d623eb4f","Type":"ContainerDied","Data":"d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd"}
Nov 01 03:10:57.522857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:57.522858    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71jq2nw"
Nov 01 03:10:57.522857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:57.522868    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d846424c663dbc44d4a062607e81b33ac390fe6a8ed9b865e2da214aa03c2bbd"
Nov 01 03:10:57.609379 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:10:57.609342    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:10:59.931755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931571    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"]
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931914    4892 cpu_manager.go:414] "containerMap: removing container" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="util"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931932    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="util"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931946    4892 cpu_manager.go:414] "containerMap: removing container" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="pull"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931954    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="pull"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931965    4892 cpu_manager.go:414] "containerMap: removing container" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="extract"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.931972    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="extract"
Nov 01 03:10:59.933255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.932092    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="def0ecae-b9e1-44ba-adc0-5070d623eb4f" containerName="extract"
Nov 01 03:10:59.935011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:10:59.934971    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.037682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.037654    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 01 03:11:00.037967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.037952    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 01 03:11:00.037967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.037961    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 01 03:11:00.057910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.057874    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 01 03:11:00.059754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.059732    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"]
Nov 01 03:11:00.107077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107038    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-manifests\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.107244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107089    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-tmp\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.107244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107185    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.107336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107248    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsflj\" (UniqueName: \"kubernetes.io/projected/ded4c32b-c7f4-4659-8567-456affd3a5ab-kube-api-access-fsflj\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.107375 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107359    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.107409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.107399    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-metrics-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.132136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.132110    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 01 03:11:00.132329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.132310    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-445ws\""
Nov 01 03:11:00.208341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208254    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsflj\" (UniqueName: \"kubernetes.io/projected/ded4c32b-c7f4-4659-8567-456affd3a5ab-kube-api-access-fsflj\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208297    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208323    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-metrics-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208356    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-manifests\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208382    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-tmp\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208412    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.208948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.208908    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-manifests\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.210854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.210818    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ded4c32b-c7f4-4659-8567-456affd3a5ab-tmp\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.211001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.210966    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-apiservice-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.211069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.211049    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-webhook-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.211112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.211087    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ded4c32b-c7f4-4659-8567-456affd3a5ab-metrics-cert\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.320339 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.319135    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsflj\" (UniqueName: \"kubernetes.io/projected/ded4c32b-c7f4-4659-8567-456affd3a5ab-kube-api-access-fsflj\") pod \"opendatahub-operator-controller-manager-576db5b58f-bnmb5\" (UID: \"ded4c32b-c7f4-4659-8567-456affd3a5ab\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:00.548967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:00.548928    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:01.104360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:01.104318    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"]
Nov 01 03:11:01.159404 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:01.159369    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podded4c32b_c7f4_4659_8567_456affd3a5ab.slice/crio-e59cf5d187811d2ccbbdb5dbce6f584de8df53d6be4018e6a36429881ee37e45 WatchSource:0}: Error finding container e59cf5d187811d2ccbbdb5dbce6f584de8df53d6be4018e6a36429881ee37e45: Status 404 returned error can't find the container with id e59cf5d187811d2ccbbdb5dbce6f584de8df53d6be4018e6a36429881ee37e45
Nov 01 03:11:01.536685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:01.536651    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5" event={"ID":"ded4c32b-c7f4-4659-8567-456affd3a5ab","Type":"ContainerStarted","Data":"e59cf5d187811d2ccbbdb5dbce6f584de8df53d6be4018e6a36429881ee37e45"}
Nov 01 03:11:05.551669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:05.551634    4892 generic.go:356] "Generic (PLEG): container finished" podID="ded4c32b-c7f4-4659-8567-456affd3a5ab" containerID="2429db9fb1f7eb72a344ac5159a7efc1851c32082dde3e3c022f240d9c3507cf" exitCode=0
Nov 01 03:11:05.552070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:05.551711    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5" event={"ID":"ded4c32b-c7f4-4659-8567-456affd3a5ab","Type":"ContainerDied","Data":"2429db9fb1f7eb72a344ac5159a7efc1851c32082dde3e3c022f240d9c3507cf"}
Nov 01 03:11:06.557338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:06.557300    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5" event={"ID":"ded4c32b-c7f4-4659-8567-456affd3a5ab","Type":"ContainerStarted","Data":"8c3ecd96cb97cd6b6cd0b7a4144ab3b409290c90f5cb959c673f5596d77e6772"}
Nov 01 03:11:06.557770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:06.557442    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:06.701666 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:06.701555    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5" podStartSLOduration=3.027913957 podStartE2EDuration="7.701540138s" totalImagesPullingTime="4.250896091s" totalInitContainerRuntime="422.73009ms" isStatefulPod=false podCreationTimestamp="2026-11-01 03:10:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:06.698690527 +0000 UTC m=+233.114590058" watchObservedRunningTime="2026-11-01 03:11:06.701540138 +0000 UTC m=+233.117439660"
Nov 01 03:11:17.564127 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:17.564092    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-576db5b58f-bnmb5"
Nov 01 03:11:18.909836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:18.909805    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"]
Nov 01 03:11:18.997062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:18.997015    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"]
Nov 01 03:11:18.997236 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:18.997069    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.060460 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.060418    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9s4v\" (UniqueName: \"kubernetes.io/projected/8ec015d2-3676-46c5-84cb-65516acb4e27-kube-api-access-d9s4v\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.060615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.060475    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ec015d2-3676-46c5-84cb-65516acb4e27-tmp\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.099856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.099829    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-96dk4\""
Nov 01 03:11:19.099955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.099876    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 01 03:11:19.099955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.099893    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 01 03:11:19.161517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.161446    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ec015d2-3676-46c5-84cb-65516acb4e27-tmp\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.161517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.161499    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9s4v\" (UniqueName: \"kubernetes.io/projected/8ec015d2-3676-46c5-84cb-65516acb4e27-kube-api-access-d9s4v\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.161835 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.161815    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8ec015d2-3676-46c5-84cb-65516acb4e27-tmp\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.222959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.222923    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9s4v\" (UniqueName: \"kubernetes.io/projected/8ec015d2-3676-46c5-84cb-65516acb4e27-kube-api-access-d9s4v\") pod \"jobset-operator-674cd59dc5-xm6tg\" (UID: \"8ec015d2-3676-46c5-84cb-65516acb4e27\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.334139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.334097    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"
Nov 01 03:11:19.726207 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:19.726177    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg"]
Nov 01 03:11:19.729861 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:19.729827    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ec015d2_3676_46c5_84cb_65516acb4e27.slice/crio-459394c99e1684e9017d63913f5aebbb621d5b8431d1cd779ad8ab929e6e07af WatchSource:0}: Error finding container 459394c99e1684e9017d63913f5aebbb621d5b8431d1cd779ad8ab929e6e07af: Status 404 returned error can't find the container with id 459394c99e1684e9017d63913f5aebbb621d5b8431d1cd779ad8ab929e6e07af
Nov 01 03:11:20.605388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:20.605352    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg" event={"ID":"8ec015d2-3676-46c5-84cb-65516acb4e27","Type":"ContainerStarted","Data":"459394c99e1684e9017d63913f5aebbb621d5b8431d1cd779ad8ab929e6e07af"}
Nov 01 03:11:23.619439 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:23.619403    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg" event={"ID":"8ec015d2-3676-46c5-84cb-65516acb4e27","Type":"ContainerStarted","Data":"e39dc2e27c2c9dc0c3843f2e1dbfabf97a5586a957a09422dcbca445480cd7a1"}
Nov 01 03:11:23.737614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:23.737513    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-xm6tg" podStartSLOduration=2.085723191 podStartE2EDuration="5.737497536s" totalImagesPullingTime="3.651774345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:23.736940512 +0000 UTC m=+250.152840042" watchObservedRunningTime="2026-11-01 03:11:23.737497536 +0000 UTC m=+250.153397064"
Nov 01 03:11:27.610955 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:11:27.610914    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:11:34.149103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.149068    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"]
Nov 01 03:11:34.151236 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.151217    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.170956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.170909    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m992\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-kube-api-access-8m992\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.171212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.170998    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.204553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.204525    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Nov 01 03:11:34.253629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.253588    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Nov 01 03:11:34.253822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.253639    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5xr7t\""
Nov 01 03:11:34.254687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.254664    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"]
Nov 01 03:11:34.272253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.272207    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m992\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-kube-api-access-8m992\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.272438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.272272    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.338401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.338360    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.338554 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.338459    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m992\" (UniqueName: \"kubernetes.io/projected/a7b525a3-688f-4c08-8b4d-527dcbf2d359-kube-api-access-8m992\") pod \"cert-manager-webhook-5dbbbf75cc-b6kpb\" (UID: \"a7b525a3-688f-4c08-8b4d-527dcbf2d359\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.461632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.461533    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:34.845114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:34.844956    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"]
Nov 01 03:11:34.847839 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:34.847808    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda7b525a3_688f_4c08_8b4d_527dcbf2d359.slice/crio-56ef341733c62d9d0ef3e4e55b97bc649b9f3abe0aebe5113edb101567933170 WatchSource:0}: Error finding container 56ef341733c62d9d0ef3e4e55b97bc649b9f3abe0aebe5113edb101567933170: Status 404 returned error can't find the container with id 56ef341733c62d9d0ef3e4e55b97bc649b9f3abe0aebe5113edb101567933170
Nov 01 03:11:35.664104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:35.664070    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb" event={"ID":"a7b525a3-688f-4c08-8b4d-527dcbf2d359","Type":"ContainerStarted","Data":"56ef341733c62d9d0ef3e4e55b97bc649b9f3abe0aebe5113edb101567933170"}
Nov 01 03:11:36.152553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.152511    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"]
Nov 01 03:11:36.155132 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.155108    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.189761 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.189730    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx7gt\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-kube-api-access-sx7gt\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.189936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.189787    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.211629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.211603    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-x4f2q\""
Nov 01 03:11:36.258993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.258952    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"]
Nov 01 03:11:36.290955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.290916    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx7gt\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-kube-api-access-sx7gt\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.291180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.291005    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.352616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.352583    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.352782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.352762    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx7gt\" (UniqueName: \"kubernetes.io/projected/fc7b944c-1b29-451a-b77a-2963f59e109a-kube-api-access-sx7gt\") pod \"cert-manager-cainjector-5c57bbd4fc-b8lkv\" (UID: \"fc7b944c-1b29-451a-b77a-2963f59e109a\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.466026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.465932    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"
Nov 01 03:11:36.854496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:36.854459    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv"]
Nov 01 03:11:36.861425 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:36.861379    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc7b944c_1b29_451a_b77a_2963f59e109a.slice/crio-017fb172298c9fed20dd4d986e650a622acea2465f69afff676d1c3c89bcfcf7 WatchSource:0}: Error finding container 017fb172298c9fed20dd4d986e650a622acea2465f69afff676d1c3c89bcfcf7: Status 404 returned error can't find the container with id 017fb172298c9fed20dd4d986e650a622acea2465f69afff676d1c3c89bcfcf7
Nov 01 03:11:37.673742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:37.673698    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv" event={"ID":"fc7b944c-1b29-451a-b77a-2963f59e109a","Type":"ContainerStarted","Data":"017fb172298c9fed20dd4d986e650a622acea2465f69afff676d1c3c89bcfcf7"}
Nov 01 03:11:39.683105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:39.683068    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb" event={"ID":"a7b525a3-688f-4c08-8b4d-527dcbf2d359","Type":"ContainerStarted","Data":"c7271998f92430b5b40b9ca4daaa8bf98d45a54108ba41dce33d1e74ceb5abf8"}
Nov 01 03:11:39.683535 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:39.683142    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:39.684627 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:39.684598    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv" event={"ID":"fc7b944c-1b29-451a-b77a-2963f59e109a","Type":"ContainerStarted","Data":"50a290e6abbd034b1858bc8411a53b4489ebd61126955716247e42e9f07c69c6"}
Nov 01 03:11:39.801627 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:39.801557    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb" podStartSLOduration=1.490353018 podStartE2EDuration="5.801537172s" totalImagesPullingTime="4.311184154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:39.800804568 +0000 UTC m=+266.216704109" watchObservedRunningTime="2026-11-01 03:11:39.801537172 +0000 UTC m=+266.217436701"
Nov 01 03:11:39.921374 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:39.921308    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-b8lkv" podStartSLOduration=1.28816759 podStartE2EDuration="3.921290554s" totalImagesPullingTime="2.633122964s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:39.920680072 +0000 UTC m=+266.336579598" watchObservedRunningTime="2026-11-01 03:11:39.921290554 +0000 UTC m=+266.337190081"
Nov 01 03:11:42.841890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.841856    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"]
Nov 01 03:11:42.843900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.843883    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:42.946990 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.946947    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75pkm\" (UniqueName: \"kubernetes.io/projected/ba648555-2f1d-4cd0-b294-89b1c432c1d2-kube-api-access-75pkm\") pod \"dns-operator-controller-manager-8557cbdb84-c778v\" (UID: \"ba648555-2f1d-4cd0-b294-89b1c432c1d2\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:42.947362 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.947338    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Nov 01 03:11:42.947512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.947339    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 01 03:11:42.947512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.947429    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hkt5c\""
Nov 01 03:11:42.947512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.947456    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 01 03:11:42.948523 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:42.948497    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"]
Nov 01 03:11:43.047862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:43.047816    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75pkm\" (UniqueName: \"kubernetes.io/projected/ba648555-2f1d-4cd0-b294-89b1c432c1d2-kube-api-access-75pkm\") pod \"dns-operator-controller-manager-8557cbdb84-c778v\" (UID: \"ba648555-2f1d-4cd0-b294-89b1c432c1d2\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:43.109313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:43.109238    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-75pkm\" (UniqueName: \"kubernetes.io/projected/ba648555-2f1d-4cd0-b294-89b1c432c1d2-kube-api-access-75pkm\") pod \"dns-operator-controller-manager-8557cbdb84-c778v\" (UID: \"ba648555-2f1d-4cd0-b294-89b1c432c1d2\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:43.155637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:43.155589    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:43.602073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:43.602037    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"]
Nov 01 03:11:43.605892 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:43.605860    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba648555_2f1d_4cd0_b294_89b1c432c1d2.slice/crio-3b016a1492fb5b7bf1de86dd0b06506e457f7365d7f3f6e9498334a7fc7aa903 WatchSource:0}: Error finding container 3b016a1492fb5b7bf1de86dd0b06506e457f7365d7f3f6e9498334a7fc7aa903: Status 404 returned error can't find the container with id 3b016a1492fb5b7bf1de86dd0b06506e457f7365d7f3f6e9498334a7fc7aa903
Nov 01 03:11:43.700280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:43.700242    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v" event={"ID":"ba648555-2f1d-4cd0-b294-89b1c432c1d2","Type":"ContainerStarted","Data":"3b016a1492fb5b7bf1de86dd0b06506e457f7365d7f3f6e9498334a7fc7aa903"}
Nov 01 03:11:44.370636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.370600    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-58c5g"]
Nov 01 03:11:44.375843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.374373    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.433117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.433079    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-58c5g"]
Nov 01 03:11:44.462385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.462339    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhwgd\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-kube-api-access-qhwgd\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.462531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.462397    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-bound-sa-token\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.478703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.478670    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-mxpmj\""
Nov 01 03:11:44.563890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.563849    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhwgd\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-kube-api-access-qhwgd\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.564080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.563903    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-bound-sa-token\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.629099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.629015    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-bound-sa-token\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.629253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.629221    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhwgd\" (UniqueName: \"kubernetes.io/projected/01e846c9-e9fc-441a-a3c6-ad7fed49b3da-kube-api-access-qhwgd\") pod \"cert-manager-597b95cb84-58c5g\" (UID: \"01e846c9-e9fc-441a-a3c6-ad7fed49b3da\") " pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:44.685384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:44.685341    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-58c5g"
Nov 01 03:11:45.058339 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:45.058287    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-58c5g"]
Nov 01 03:11:45.061757 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:45.061728    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01e846c9_e9fc_441a_a3c6_ad7fed49b3da.slice/crio-f802a4884104807a171349cc07ecbaf2c1521595b18021454a19c541d5998a25 WatchSource:0}: Error finding container f802a4884104807a171349cc07ecbaf2c1521595b18021454a19c541d5998a25: Status 404 returned error can't find the container with id f802a4884104807a171349cc07ecbaf2c1521595b18021454a19c541d5998a25
Nov 01 03:11:45.691235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:45.691204    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb"
Nov 01 03:11:45.709408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:45.709372    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-58c5g" event={"ID":"01e846c9-e9fc-441a-a3c6-ad7fed49b3da","Type":"ContainerStarted","Data":"f096f3f220aedd6eaeec1cd8cc2fa365bb433cacbffe334671543d80d876749a"}
Nov 01 03:11:45.709408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:45.709414    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-58c5g" event={"ID":"01e846c9-e9fc-441a-a3c6-ad7fed49b3da","Type":"ContainerStarted","Data":"f802a4884104807a171349cc07ecbaf2c1521595b18021454a19c541d5998a25"}
Nov 01 03:11:47.779691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.779639    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-58c5g" podStartSLOduration=3.779621246 podStartE2EDuration="3.779621246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:45.97742451 +0000 UTC m=+272.393324041" watchObservedRunningTime="2026-11-01 03:11:47.779621246 +0000 UTC m=+274.195520777"
Nov 01 03:11:47.780160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.780141    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"]
Nov 01 03:11:47.782221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.782204    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.887665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.887617    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b66cn\""
Nov 01 03:11:47.887665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.887645    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Nov 01 03:11:47.891183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.891156    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Nov 01 03:11:47.891183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.891165    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 01 03:11:47.891427 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.891170    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 01 03:11:47.892173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.892143    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/205d2d1e-a469-47b4-b517-13b220fe8c3f-tmp\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.892313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.892188    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205d2d1e-a469-47b4-b517-13b220fe8c3f-tls-certs\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.892417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.892398    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cn7j\" (UniqueName: \"kubernetes.io/projected/205d2d1e-a469-47b4-b517-13b220fe8c3f-kube-api-access-2cn7j\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.892993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.892959    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"]
Nov 01 03:11:47.993472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.993434    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cn7j\" (UniqueName: \"kubernetes.io/projected/205d2d1e-a469-47b4-b517-13b220fe8c3f-kube-api-access-2cn7j\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.993472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.993489    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/205d2d1e-a469-47b4-b517-13b220fe8c3f-tmp\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.993696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.993508    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205d2d1e-a469-47b4-b517-13b220fe8c3f-tls-certs\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.995771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.995740    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/205d2d1e-a469-47b4-b517-13b220fe8c3f-tmp\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:47.996093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:47.996070    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/205d2d1e-a469-47b4-b517-13b220fe8c3f-tls-certs\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:48.054801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.054730    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cn7j\" (UniqueName: \"kubernetes.io/projected/205d2d1e-a469-47b4-b517-13b220fe8c3f-kube-api-access-2cn7j\") pod \"kube-auth-proxy-7d5d9754f-pzp6s\" (UID: \"205d2d1e-a469-47b4-b517-13b220fe8c3f\") " pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:48.116654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.116614    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"
Nov 01 03:11:48.490187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.490104    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s"]
Nov 01 03:11:48.493763 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:48.493738    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205d2d1e_a469_47b4_b517_13b220fe8c3f.slice/crio-90e3218747c8466da6f11e244794e59e08f84adb90c4fcbcb31c88edb2aeee20 WatchSource:0}: Error finding container 90e3218747c8466da6f11e244794e59e08f84adb90c4fcbcb31c88edb2aeee20: Status 404 returned error can't find the container with id 90e3218747c8466da6f11e244794e59e08f84adb90c4fcbcb31c88edb2aeee20
Nov 01 03:11:48.725110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.725073    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v" event={"ID":"ba648555-2f1d-4cd0-b294-89b1c432c1d2","Type":"ContainerStarted","Data":"0efd0473fcf44cd7fe7fb403c9939506900351e6fcc8c7d73ffdd79bebcf6e6d"}
Nov 01 03:11:48.725300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.725174    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:11:48.726289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.726264    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s" event={"ID":"205d2d1e-a469-47b4-b517-13b220fe8c3f","Type":"ContainerStarted","Data":"90e3218747c8466da6f11e244794e59e08f84adb90c4fcbcb31c88edb2aeee20"}
Nov 01 03:11:48.844228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:48.844157    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v" podStartSLOduration=2.675099951 podStartE2EDuration="6.844137939s" totalImagesPullingTime="4.169037988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:48.843352007 +0000 UTC m=+275.259251538" watchObservedRunningTime="2026-11-01 03:11:48.844137939 +0000 UTC m=+275.260037469"
Nov 01 03:11:51.740213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:51.740112    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s" event={"ID":"205d2d1e-a469-47b4-b517-13b220fe8c3f","Type":"ContainerStarted","Data":"fbb38fa5619afa25b677608303cb3399348d983d64dba34fbcbf857a9dd31344"}
Nov 01 03:11:51.857466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:51.857403    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d5d9754f-pzp6s" podStartSLOduration=2.057264951 podStartE2EDuration="4.857384631s" totalImagesPullingTime="2.80011968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:11:51.856344448 +0000 UTC m=+278.272243978" watchObservedRunningTime="2026-11-01 03:11:51.857384631 +0000 UTC m=+278.273284161"
Nov 01 03:11:57.610476 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:11:57.610439    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:11:58.049856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.049821    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:11:58.056992 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.056959    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.111514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.111474    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:11:58.160034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.159996    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 01 03:11:58.160034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.160030    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-sr5bv\""
Nov 01 03:11:58.179632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.179590    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj9qf\" (UniqueName: \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.179632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.179632    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.280506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.280474    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj9qf\" (UniqueName: \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.280672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.280512    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.281229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.281209    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.299176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.299141    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:11:58.342425 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.342349    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj9qf\" (UniqueName: \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\") pod \"limitador-limitador-74dfcfb894-x228s\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.369632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.369592    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:11:58.749195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.749124    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:11:58.752281 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:11:58.752254    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405aecd8_cb23_4a13_8705_5fd8dca04eaf.slice/crio-8d1d0146cb6a127d4a3776f95e5422153c8fc7f7ce7fa7e9612ab9694c3105ef WatchSource:0}: Error finding container 8d1d0146cb6a127d4a3776f95e5422153c8fc7f7ce7fa7e9612ab9694c3105ef: Status 404 returned error can't find the container with id 8d1d0146cb6a127d4a3776f95e5422153c8fc7f7ce7fa7e9612ab9694c3105ef
Nov 01 03:11:58.769504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:58.769467    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" event={"ID":"405aecd8-cb23-4a13-8705-5fd8dca04eaf","Type":"ContainerStarted","Data":"8d1d0146cb6a127d4a3776f95e5422153c8fc7f7ce7fa7e9612ab9694c3105ef"}
Nov 01 03:11:59.733157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:11:59.733119    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-c778v"
Nov 01 03:12:01.781478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:01.781442    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" event={"ID":"405aecd8-cb23-4a13-8705-5fd8dca04eaf","Type":"ContainerStarted","Data":"8f8cdcee5cd96ce2a0ce2d06098ed24428aa70dc341de438cd4de1fd525003fa"}
Nov 01 03:12:01.781904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:01.781529    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:12:01.908603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:01.908542    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" podStartSLOduration=1.657618432 podStartE2EDuration="3.908526384s" totalImagesPullingTime="2.250907952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:12:01.907793574 +0000 UTC m=+288.323693115" watchObservedRunningTime="2026-11-01 03:12:01.908526384 +0000 UTC m=+288.324425918"
Nov 01 03:12:02.547446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.547410    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"]
Nov 01 03:12:02.550756 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.550732    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.608363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.608327    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"]
Nov 01 03:12:02.653316 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.653280    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Nov 01 03:12:02.653316 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.653280    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Nov 01 03:12:02.722673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.722630    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33f126-741c-4515-842e-a9554a1597d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.722840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.722742    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b33f126-741c-4515-842e-a9554a1597d0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.722840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.722769    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt6qq\" (UniqueName: \"kubernetes.io/projected/9b33f126-741c-4515-842e-a9554a1597d0-kube-api-access-pt6qq\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.823758 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.823666    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b33f126-741c-4515-842e-a9554a1597d0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.823758 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.823710    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt6qq\" (UniqueName: \"kubernetes.io/projected/9b33f126-741c-4515-842e-a9554a1597d0-kube-api-access-pt6qq\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.824195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.823768    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33f126-741c-4515-842e-a9554a1597d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.824392 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.824360    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9b33f126-741c-4515-842e-a9554a1597d0-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.826250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.826224    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9b33f126-741c-4515-842e-a9554a1597d0-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:02.885389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:02.885350    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt6qq\" (UniqueName: \"kubernetes.io/projected/9b33f126-741c-4515-842e-a9554a1597d0-kube-api-access-pt6qq\") pod \"kuadrant-console-plugin-66c4fb575d-8mskm\" (UID: \"9b33f126-741c-4515-842e-a9554a1597d0\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:03.162015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:03.161920    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"
Nov 01 03:12:03.532233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:03.532202    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm"]
Nov 01 03:12:03.535605 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:12:03.535577    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b33f126_741c_4515_842e_a9554a1597d0.slice/crio-74d8530de372c4bb0d66651f41d5c6666a627f23a2a1cce8995993286bb94501 WatchSource:0}: Error finding container 74d8530de372c4bb0d66651f41d5c6666a627f23a2a1cce8995993286bb94501: Status 404 returned error can't find the container with id 74d8530de372c4bb0d66651f41d5c6666a627f23a2a1cce8995993286bb94501
Nov 01 03:12:03.790738 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:03.790648    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm" event={"ID":"9b33f126-741c-4515-842e-a9554a1597d0","Type":"ContainerStarted","Data":"74d8530de372c4bb0d66651f41d5c6666a627f23a2a1cce8995993286bb94501"}
Nov 01 03:12:10.826038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:10.825992    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm" event={"ID":"9b33f126-741c-4515-842e-a9554a1597d0","Type":"ContainerStarted","Data":"ee6eaccd67bc4e1db64354b98673c9f1d236e5e26b900852ea509f0497fd3288"}
Nov 01 03:12:10.942453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:10.942395    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-8mskm" podStartSLOduration=2.654865042 podStartE2EDuration="8.942379236s" totalImagesPullingTime="6.287514194s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:12:10.940915025 +0000 UTC m=+297.356814547" watchObservedRunningTime="2026-11-01 03:12:10.942379236 +0000 UTC m=+297.358278764"
Nov 01 03:12:12.786582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:12.786545    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:12:13.715025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:13.714998    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:12:13.715439 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:13.715416    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:12:18.220385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:18.220350    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:12:18.220844 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:18.220551    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" podUID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" containerName="limitador" containerID="cri-o://8f8cdcee5cd96ce2a0ce2d06098ed24428aa70dc341de438cd4de1fd525003fa" gracePeriod=30
Nov 01 03:12:18.860018 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:18.859905    4892 generic.go:356] "Generic (PLEG): container finished" podID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" containerID="8f8cdcee5cd96ce2a0ce2d06098ed24428aa70dc341de438cd4de1fd525003fa" exitCode=0
Nov 01 03:12:18.860018 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:18.859955    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" event={"ID":"405aecd8-cb23-4a13-8705-5fd8dca04eaf","Type":"ContainerDied","Data":"8f8cdcee5cd96ce2a0ce2d06098ed24428aa70dc341de438cd4de1fd525003fa"}
Nov 01 03:12:19.162962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.162933    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:12:19.261321 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.261284    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\" (UniqueName: \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\") pod \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") "
Nov 01 03:12:19.261798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.261355    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\" (UniqueName: \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\") pod \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\" (UID: \"405aecd8-cb23-4a13-8705-5fd8dca04eaf\") "
Nov 01 03:12:19.261798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.261722    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file" pod "405aecd8-cb23-4a13-8705-5fd8dca04eaf" (UID: "405aecd8-cb23-4a13-8705-5fd8dca04eaf"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:12:19.263635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.263606    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf" pod "405aecd8-cb23-4a13-8705-5fd8dca04eaf" (UID: "405aecd8-cb23-4a13-8705-5fd8dca04eaf"). InnerVolumeSpecName "kube-api-access-hj9qf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:12:19.362798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.362748    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj9qf\" (UniqueName: \"kubernetes.io/projected/405aecd8-cb23-4a13-8705-5fd8dca04eaf-kube-api-access-hj9qf\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:12:19.362798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.362793    4892 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/405aecd8-cb23-4a13-8705-5fd8dca04eaf-config-file\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:12:19.865074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.865036    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s" event={"ID":"405aecd8-cb23-4a13-8705-5fd8dca04eaf","Type":"ContainerDied","Data":"8d1d0146cb6a127d4a3776f95e5422153c8fc7f7ce7fa7e9612ab9694c3105ef"}
Nov 01 03:12:19.865225 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.865082    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-x228s"
Nov 01 03:12:19.865225 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:19.865099    4892 scope.go:122] "RemoveContainer" containerID="8f8cdcee5cd96ce2a0ce2d06098ed24428aa70dc341de438cd4de1fd525003fa"
Nov 01 03:12:20.028739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:20.028696    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:12:20.078166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:20.078134    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-x228s"]
Nov 01 03:12:21.699525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:21.699490    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" path="/var/lib/kubelet/pods/405aecd8-cb23-4a13-8705-5fd8dca04eaf/volumes"
Nov 01 03:12:24.095606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.095569    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"]
Nov 01 03:12:24.096022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.095859    4892 cpu_manager.go:414] "containerMap: removing container" podUID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" containerName="limitador"
Nov 01 03:12:24.096022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.095871    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" containerName="limitador"
Nov 01 03:12:24.096022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.095942    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="405aecd8-cb23-4a13-8705-5fd8dca04eaf" containerName="limitador"
Nov 01 03:12:24.097564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.097547    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.173956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.173926    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 01 03:12:24.174138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.173993    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-9lmk4\""
Nov 01 03:12:24.197177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.197140    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98wmw\" (UniqueName: \"kubernetes.io/projected/92e6cb09-501b-4ae9-96a5-f5b01e520468-kube-api-access-98wmw\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.197371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.197291    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/92e6cb09-501b-4ae9-96a5-f5b01e520468-data\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.203448 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.203424    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"]
Nov 01 03:12:24.219698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.219672    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 01 03:12:24.223740 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.223721    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 01 03:12:24.298322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.298273    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/92e6cb09-501b-4ae9-96a5-f5b01e520468-data\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.298492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.298335    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98wmw\" (UniqueName: \"kubernetes.io/projected/92e6cb09-501b-4ae9-96a5-f5b01e520468-kube-api-access-98wmw\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.298725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.298705    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/92e6cb09-501b-4ae9-96a5-f5b01e520468-data\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.359827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.359748    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98wmw\" (UniqueName: \"kubernetes.io/projected/92e6cb09-501b-4ae9-96a5-f5b01e520468-kube-api-access-98wmw\") pod \"postgres-7cdbf6dc44-cv8c7\" (UID: \"92e6cb09-501b-4ae9-96a5-f5b01e520468\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.408585 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.408545    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:24.779714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.779683    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"]
Nov 01 03:12:24.782656 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:12:24.782620    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92e6cb09_501b_4ae9_96a5_f5b01e520468.slice/crio-362b17ab9ca5d082f2616be2f01772197034148052508a8f8757ba78ed265bf4 WatchSource:0}: Error finding container 362b17ab9ca5d082f2616be2f01772197034148052508a8f8757ba78ed265bf4: Status 404 returned error can't find the container with id 362b17ab9ca5d082f2616be2f01772197034148052508a8f8757ba78ed265bf4
Nov 01 03:12:24.784162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.784139    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:12:24.888606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:24.888570    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7" event={"ID":"92e6cb09-501b-4ae9-96a5-f5b01e520468","Type":"ContainerStarted","Data":"362b17ab9ca5d082f2616be2f01772197034148052508a8f8757ba78ed265bf4"}
Nov 01 03:12:27.622397 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:27.622354    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:12:31.921795 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:31.921753    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7" event={"ID":"92e6cb09-501b-4ae9-96a5-f5b01e520468","Type":"ContainerStarted","Data":"a5676d1535c7e4f9b7cc00d74ae506706f4fb8df7a70fa2a3c6ad4069d0b44e5"}
Nov 01 03:12:31.922718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:31.922692    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:32.038264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:32.038198    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7" podStartSLOduration=1.36390826 podStartE2EDuration="8.038183378s" totalImagesPullingTime="6.674275118s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:12:32.037270253 +0000 UTC m=+318.453169783" watchObservedRunningTime="2026-11-01 03:12:32.038183378 +0000 UTC m=+318.454082915"
Nov 01 03:12:37.956310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:37.956279    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-cv8c7"
Nov 01 03:12:44.950678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:44.950645    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:12:44.974280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:44.974253    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.027020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.026966    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 01 03:12:45.027215 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.027145    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 01 03:12:45.076533 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076496    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 01 03:12:45.076533 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076493    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 01 03:12:45.076784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076528    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 01 03:12:45.076784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076599    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-kdlhc\""
Nov 01 03:12:45.076784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076770    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm96d\" (UniqueName: \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.076908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076873    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.076908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.076905    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.077870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.077851    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:12:45.178306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.178261    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm96d\" (UniqueName: \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.178522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.178349    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.178522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.178393    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.178638 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:45.178561    4892 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Nov 01 03:12:45.178689 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:45.178638    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs podName:2935b9bd-45be-4a05-826d-f426bb17a06b nodeName:}" failed. No retries permitted until 2026-11-01 03:12:45.67861687 +0000 UTC m=+332.094516398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs") pod "maas-controller-579cf7c6d-8ggcz" (UID: "2935b9bd-45be-4a05-826d-f426bb17a06b") : secret "maas-controller-metrics-tls" not found
Nov 01 03:12:45.180995 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.180953    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.242048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.241961    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm96d\" (UniqueName: \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.378528 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.378491    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:12:45.381925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.381902    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.441646 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.441601    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:12:45.480543 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.480511    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.480712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.480567    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.480712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.480675    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m874b\" (UniqueName: \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.581907 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.581869    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.582111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.581926    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.582111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.581957    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m874b\" (UniqueName: \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.584579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.584547    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.584694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.584547    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.643500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.643462    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m874b\" (UniqueName: \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\") pod \"maas-controller-5d96cbcb67-wbhf8\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") " pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.682566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.682521    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.684933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.684913    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") pod \"maas-controller-579cf7c6d-8ggcz\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") " pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:45.695229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.695196    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:45.886321 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:45.886228    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:46.069225 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:46.069182    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:12:46.102305 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:12:46.102254    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7934c60_939d_444e_a9cc_d61a30c84adf.slice/crio-c5716dc64d272d8f046e7f06a4c91cd5090b826dbda31db4e01e9629c6fbf2ee WatchSource:0}: Error finding container c5716dc64d272d8f046e7f06a4c91cd5090b826dbda31db4e01e9629c6fbf2ee: Status 404 returned error can't find the container with id c5716dc64d272d8f046e7f06a4c91cd5090b826dbda31db4e01e9629c6fbf2ee
Nov 01 03:12:46.265213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:46.265180    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:12:46.268142 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:12:46.268110    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2935b9bd_45be_4a05_826d_f426bb17a06b.slice/crio-2af49e0b9563349fc5e1220025414b1b97093a57d9d073893f4f132e9312c4e8 WatchSource:0}: Error finding container 2af49e0b9563349fc5e1220025414b1b97093a57d9d073893f4f132e9312c4e8: Status 404 returned error can't find the container with id 2af49e0b9563349fc5e1220025414b1b97093a57d9d073893f4f132e9312c4e8
Nov 01 03:12:46.985142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:46.985099    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerStarted","Data":"c5716dc64d272d8f046e7f06a4c91cd5090b826dbda31db4e01e9629c6fbf2ee"}
Nov 01 03:12:46.986205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:46.986181    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" event={"ID":"2935b9bd-45be-4a05-826d-f426bb17a06b","Type":"ContainerStarted","Data":"2af49e0b9563349fc5e1220025414b1b97093a57d9d073893f4f132e9312c4e8"}
Nov 01 03:12:50.003446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.003414    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerStarted","Data":"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"}
Nov 01 03:12:50.003893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.003552    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:12:50.005119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.005091    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" event={"ID":"2935b9bd-45be-4a05-826d-f426bb17a06b","Type":"ContainerStarted","Data":"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"}
Nov 01 03:12:50.005224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.005184    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:12:50.125223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.125160    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" podStartSLOduration=1.630351771 podStartE2EDuration="5.125139521s" totalImagesPullingTime="3.49478775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:12:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:12:50.123896629 +0000 UTC m=+336.539796179" watchObservedRunningTime="2026-11-01 03:12:50.125139521 +0000 UTC m=+336.541039047"
Nov 01 03:12:50.245741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:50.245681    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" podStartSLOduration=2.873942665 podStartE2EDuration="6.245665418s" totalImagesPullingTime="3.371722753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:12:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:12:50.244931298 +0000 UTC m=+336.660830827" watchObservedRunningTime="2026-11-01 03:12:50.245665418 +0000 UTC m=+336.661564946"
Nov 01 03:12:53.002927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.002889    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"]
Nov 01 03:12:53.005989 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.005955    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.108313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.108225    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 01 03:12:53.108313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.108253    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 01 03:12:53.108313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.108263    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 01 03:12:53.108313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.108266    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-wsknt\""
Nov 01 03:12:53.109609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.109589    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"]
Nov 01 03:12:53.146245 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.146211    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.146440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.146262    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mvbq\" (UniqueName: \"kubernetes.io/projected/80c5ac4d-710e-44a9-9394-0de4e216d06c-kube-api-access-2mvbq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.146440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.146372    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.146440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.146414    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.247685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.247643    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.247863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.247693    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.247863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.247740    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.247863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.247769    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mvbq\" (UniqueName: \"kubernetes.io/projected/80c5ac4d-710e-44a9-9394-0de4e216d06c-kube-api-access-2mvbq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.247863 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:53.247809    4892 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Nov 01 03:12:53.247863 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:53.247850    4892 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Nov 01 03:12:53.248063 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:53.247876    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert podName:80c5ac4d-710e-44a9-9394-0de4e216d06c nodeName:}" failed. No retries permitted until 2026-11-01 03:12:53.747858214 +0000 UTC m=+340.163757734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-sq6bx" (UID: "80c5ac4d-710e-44a9-9394-0de4e216d06c") : secret "usage-logs-tenancy-proxy-tls" not found
Nov 01 03:12:53.248063 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:53.247925    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source podName:80c5ac4d-710e-44a9-9394-0de4e216d06c nodeName:}" failed. No retries permitted until 2026-11-01 03:12:53.747909895 +0000 UTC m=+340.163809421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-sq6bx" (UID: "80c5ac4d-710e-44a9-9394-0de4e216d06c") : configmap "usage-tenancy-proxy-source" not found
Nov 01 03:12:53.248485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.248465    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.309363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.309288    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mvbq\" (UniqueName: \"kubernetes.io/projected/80c5ac4d-710e-44a9-9394-0de4e216d06c-kube-api-access-2mvbq\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.751946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.751852    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.751946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.751921    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.752699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.752677    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/80c5ac4d-710e-44a9-9394-0de4e216d06c-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.754306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.754277    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/80c5ac4d-710e-44a9-9394-0de4e216d06c-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-sq6bx\" (UID: \"80c5ac4d-710e-44a9-9394-0de4e216d06c\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:53.918341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:53.918299    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:12:54.304850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:54.304762    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"]
Nov 01 03:12:54.307499 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:12:54.307475    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c5ac4d_710e_44a9_9394_0de4e216d06c.slice/crio-c392f6ece4624e147e8151c7178a77defcfd522fe994a99c25e7567b5f3b791f WatchSource:0}: Error finding container c392f6ece4624e147e8151c7178a77defcfd522fe994a99c25e7567b5f3b791f: Status 404 returned error can't find the container with id c392f6ece4624e147e8151c7178a77defcfd522fe994a99c25e7567b5f3b791f
Nov 01 03:12:55.025969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:12:55.025932    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx" event={"ID":"80c5ac4d-710e-44a9-9394-0de4e216d06c","Type":"ContainerStarted","Data":"c392f6ece4624e147e8151c7178a77defcfd522fe994a99c25e7567b5f3b791f"}
Nov 01 03:12:57.621330 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:12:57.621288    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:13:01.013003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:01.012535    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:13:01.013463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:01.012973    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:13:01.183867 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:01.183830    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:13:01.184108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:01.184060    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" podUID="2935b9bd-45be-4a05-826d-f426bb17a06b" containerName="manager" containerID="cri-o://a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141" gracePeriod=30
Nov 01 03:13:01.989369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:01.989329    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:13:02.063434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.063395    4892 generic.go:356] "Generic (PLEG): container finished" podID="2935b9bd-45be-4a05-826d-f426bb17a06b" containerID="a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141" exitCode=0
Nov 01 03:13:02.063875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.063524    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" event={"ID":"2935b9bd-45be-4a05-826d-f426bb17a06b","Type":"ContainerDied","Data":"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"}
Nov 01 03:13:02.063875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.063579    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-8ggcz" event={"ID":"2935b9bd-45be-4a05-826d-f426bb17a06b","Type":"ContainerDied","Data":"2af49e0b9563349fc5e1220025414b1b97093a57d9d073893f4f132e9312c4e8"}
Nov 01 03:13:02.063875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.063607    4892 scope.go:122] "RemoveContainer" containerID="a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"
Nov 01 03:13:02.063875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.063749    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-8ggcz"
Nov 01 03:13:02.072885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.072859    4892 scope.go:122] "RemoveContainer" containerID="a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"
Nov 01 03:13:02.073267 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:13:02.073235    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141\": container with ID starting with a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141 not found: ID does not exist" containerID="a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"
Nov 01 03:13:02.073374 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.073274    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141"} err="failed to get container status \"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141\": rpc error: code = NotFound desc = could not find container \"a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141\": container with ID starting with a3bba2b45647d74c2d99296b08304faf263482de4be03a79fd11585b5552b141 not found: ID does not exist"
Nov 01 03:13:02.129540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.129462    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") pod \"2935b9bd-45be-4a05-826d-f426bb17a06b\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") "
Nov 01 03:13:02.129694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.129595    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\") pod \"2935b9bd-45be-4a05-826d-f426bb17a06b\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") "
Nov 01 03:13:02.129694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.129652    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\" (UniqueName: \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\") pod \"2935b9bd-45be-4a05-826d-f426bb17a06b\" (UID: \"2935b9bd-45be-4a05-826d-f426bb17a06b\") "
Nov 01 03:13:02.132156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.132114    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert" pod "2935b9bd-45be-4a05-826d-f426bb17a06b" (UID: "2935b9bd-45be-4a05-826d-f426bb17a06b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:13:02.132301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.132117    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d" pod "2935b9bd-45be-4a05-826d-f426bb17a06b" (UID: "2935b9bd-45be-4a05-826d-f426bb17a06b"). InnerVolumeSpecName "kube-api-access-wm96d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:13:02.132301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.132193    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs" pod "2935b9bd-45be-4a05-826d-f426bb17a06b" (UID: "2935b9bd-45be-4a05-826d-f426bb17a06b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:13:02.230776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.230738    4892 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-webhook-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:02.230776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.230777    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm96d\" (UniqueName: \"kubernetes.io/projected/2935b9bd-45be-4a05-826d-f426bb17a06b-kube-api-access-wm96d\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:02.230993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.230792    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2935b9bd-45be-4a05-826d-f426bb17a06b-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:02.486510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.486428    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:13:02.490971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:02.490943    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-8ggcz"]
Nov 01 03:13:03.700755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:03.700716    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2935b9bd-45be-4a05-826d-f426bb17a06b" path="/var/lib/kubelet/pods/2935b9bd-45be-4a05-826d-f426bb17a06b/volumes"
Nov 01 03:13:07.090276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:07.090237    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx" event={"ID":"80c5ac4d-710e-44a9-9394-0de4e216d06c","Type":"ContainerStarted","Data":"f0dd8538f7173f784143636d9d54dc8d2445932dfcb08c658df98c46bb79bcef"}
Nov 01 03:13:07.090682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:07.090315    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:13:07.231770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:07.231707    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx" podStartSLOduration=2.827341494 podStartE2EDuration="15.231691829s" totalImagesPullingTime="12.404350335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:13:07.229451927 +0000 UTC m=+353.645351455" watchObservedRunningTime="2026-11-01 03:13:07.231691829 +0000 UTC m=+353.647591358"
Nov 01 03:13:18.100596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:18.100560    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-sq6bx"
Nov 01 03:13:21.609325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.609290    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-p8ncj"]
Nov 01 03:13:21.609845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.609777    4892 cpu_manager.go:414] "containerMap: removing container" podUID="2935b9bd-45be-4a05-826d-f426bb17a06b" containerName="manager"
Nov 01 03:13:21.609845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.609796    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2935b9bd-45be-4a05-826d-f426bb17a06b" containerName="manager"
Nov 01 03:13:21.609957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.609878    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="2935b9bd-45be-4a05-826d-f426bb17a06b" containerName="manager"
Nov 01 03:13:21.617837 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.617802    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.671723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.671683    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-p8ncj"]
Nov 01 03:13:21.688599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.688560    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-cert\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.688804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.688717    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56lcv\" (UniqueName: \"kubernetes.io/projected/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-kube-api-access-56lcv\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.724395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.724361    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Nov 01 03:13:21.724395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.724363    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-pk8s2\""
Nov 01 03:13:21.770422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.770385    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:13:21.774130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.774100    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.789212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.789178    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56lcv\" (UniqueName: \"kubernetes.io/projected/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-kube-api-access-56lcv\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.789504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.789219    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-cert\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.789504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.789258    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.789504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.789324    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vhsf\" (UniqueName: \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.791723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.791697    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-cert\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.822013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.821965    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:13:21.841569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.841529    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9qqrj\""
Nov 01 03:13:21.841737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.841639    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 01 03:13:21.876191 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.876116    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56lcv\" (UniqueName: \"kubernetes.io/projected/b8a0b910-eb4e-4ace-bc8c-cad858b4c819-kube-api-access-56lcv\") pod \"kserve-controller-manager-776cc66d47-p8ncj\" (UID: \"b8a0b910-eb4e-4ace-bc8c-cad858b4c819\") " pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.890236 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.890194    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vhsf\" (UniqueName: \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.890424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.890285    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.892696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.892676    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:21.931405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.931329    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:21.968333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:21.968291    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vhsf\" (UniqueName: \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\") pod \"llmisvc-controller-manager-869d5dcc5d-n2hxj\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:22.089470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:22.089433    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:22.305828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:22.305804    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-p8ncj"]
Nov 01 03:13:22.308905 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:13:22.308877    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a0b910_eb4e_4ace_bc8c_cad858b4c819.slice/crio-1f38099ec3b45c35ea46d18175ce21bbf7462bc340cce4e13f8f240fad9abfac WatchSource:0}: Error finding container 1f38099ec3b45c35ea46d18175ce21bbf7462bc340cce4e13f8f240fad9abfac: Status 404 returned error can't find the container with id 1f38099ec3b45c35ea46d18175ce21bbf7462bc340cce4e13f8f240fad9abfac
Nov 01 03:13:22.466037 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:22.465841    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:13:22.468591 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:13:22.468552    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod59122762_6f85_4f60_9881_1c193087cfbe.slice/crio-a3b89df35ec57a34d8e1cff803bf0973577b83ec0408307a157bfd5dc2717ec7 WatchSource:0}: Error finding container a3b89df35ec57a34d8e1cff803bf0973577b83ec0408307a157bfd5dc2717ec7: Status 404 returned error can't find the container with id a3b89df35ec57a34d8e1cff803bf0973577b83ec0408307a157bfd5dc2717ec7
Nov 01 03:13:23.158643 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:23.158605    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj" event={"ID":"b8a0b910-eb4e-4ace-bc8c-cad858b4c819","Type":"ContainerStarted","Data":"1f38099ec3b45c35ea46d18175ce21bbf7462bc340cce4e13f8f240fad9abfac"}
Nov 01 03:13:23.159863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:23.159837    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" event={"ID":"59122762-6f85-4f60-9881-1c193087cfbe","Type":"ContainerStarted","Data":"a3b89df35ec57a34d8e1cff803bf0973577b83ec0408307a157bfd5dc2717ec7"}
Nov 01 03:13:24.994110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:24.994075    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:24.997498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:24.997473    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.016910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.016825    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpjq2\" (UniqueName: \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.016910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.016887    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.017170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.017025    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.017170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.017081    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.102606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.102571    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zx9nh\""
Nov 01 03:13:25.102791 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.102619    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 01 03:13:25.102791 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.102621    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 01 03:13:25.104193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.104167    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:25.118350 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.118292    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpjq2\" (UniqueName: \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.118548 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.118363    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.118548 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.118389    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.118548 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.118425    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.119217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.119189    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.121120 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.121093    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.121244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.121215    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.172098 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.172058    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj" event={"ID":"b8a0b910-eb4e-4ace-bc8c-cad858b4c819","Type":"ContainerStarted","Data":"91cb7ad4344945e9172504f704ea46ab75c97d1a740190c375be5bff23a60205"}
Nov 01 03:13:25.172283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.172179    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:25.213385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.213349    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpjq2\" (UniqueName: \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\") pod \"maas-api-6d669c7fcd-zqlpr\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") " pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.292269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.291178    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj" podStartSLOduration=1.839882626 podStartE2EDuration="4.291158906s" totalImagesPullingTime="2.45127628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:13:25.28846783 +0000 UTC m=+371.704367359" watchObservedRunningTime="2026-11-01 03:13:25.291158906 +0000 UTC m=+371.707058439"
Nov 01 03:13:25.310719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.310661    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:25.699500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:25.699473    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:25.723350 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:13:25.723304    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9935cbf0_95c6_412d_b2c7_e3fedd37e7a1.slice/crio-8e6d433c54b8deede0b1bfbe11d6eaea391bd8144e7fd0521d535e0bcbf24681 WatchSource:0}: Error finding container 8e6d433c54b8deede0b1bfbe11d6eaea391bd8144e7fd0521d535e0bcbf24681: Status 404 returned error can't find the container with id 8e6d433c54b8deede0b1bfbe11d6eaea391bd8144e7fd0521d535e0bcbf24681
Nov 01 03:13:26.177891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:26.177843    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" event={"ID":"59122762-6f85-4f60-9881-1c193087cfbe","Type":"ContainerStarted","Data":"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"}
Nov 01 03:13:26.178335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:26.178076    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:26.179150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:26.179125    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" event={"ID":"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1","Type":"ContainerStarted","Data":"8e6d433c54b8deede0b1bfbe11d6eaea391bd8144e7fd0521d535e0bcbf24681"}
Nov 01 03:13:26.294830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:26.294774    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" podStartSLOduration=2.156778443 podStartE2EDuration="5.294757089s" totalImagesPullingTime="3.137978646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:13:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:13:26.293786694 +0000 UTC m=+372.709686225" watchObservedRunningTime="2026-11-01 03:13:26.294757089 +0000 UTC m=+372.710656645"
Nov 01 03:13:27.612198 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:13:27.612161    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:13:28.190590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:28.190552    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" event={"ID":"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1","Type":"ContainerStarted","Data":"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"}
Nov 01 03:13:28.190768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:28.190640    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:28.309476 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:28.309418    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" podStartSLOduration=2.419982531 podStartE2EDuration="4.309400753s" totalImagesPullingTime="1.889418222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:13:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:13:28.307846635 +0000 UTC m=+374.723746163" watchObservedRunningTime="2026-11-01 03:13:28.309400753 +0000 UTC m=+374.725300313"
Nov 01 03:13:34.201076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:34.201041    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:46.706725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:46.706685    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:46.707203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:46.706906    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" podUID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" containerName="maas-api" containerID="cri-o://163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e" gracePeriod=30
Nov 01 03:13:46.961644 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:46.961580    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:47.017534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.017495    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\" (UniqueName: \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\") pod \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") "
Nov 01 03:13:47.017707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.017545    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\") pod \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") "
Nov 01 03:13:47.017707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.017592    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\") pod \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") "
Nov 01 03:13:47.017707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.017621    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\") pod \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\" (UID: \"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1\") "
Nov 01 03:13:47.017943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.017919    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca" pod "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" (UID: "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:13:47.019970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.019930    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs" pod "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" (UID: "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:13:47.019970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.019950    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2" pod "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" (UID: "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1"). InnerVolumeSpecName "kube-api-access-kpjq2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:13:47.020139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.019975    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls" pod "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" (UID: "9935cbf0-95c6-412d-b2c7-e3fedd37e7a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:13:47.118687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.118653    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kpjq2\" (UniqueName: \"kubernetes.io/projected/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-kube-api-access-kpjq2\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:47.118687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.118689    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:47.118968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.118701    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:47.118968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.118710    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:13:47.275809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.275771    4892 generic.go:356] "Generic (PLEG): container finished" podID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" containerID="163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e" exitCode=0
Nov 01 03:13:47.276006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.275823    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"
Nov 01 03:13:47.276006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.275860    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" event={"ID":"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1","Type":"ContainerDied","Data":"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"}
Nov 01 03:13:47.276006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.275922    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr" event={"ID":"9935cbf0-95c6-412d-b2c7-e3fedd37e7a1","Type":"ContainerDied","Data":"8e6d433c54b8deede0b1bfbe11d6eaea391bd8144e7fd0521d535e0bcbf24681"}
Nov 01 03:13:47.276006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.275951    4892 scope.go:122] "RemoveContainer" containerID="163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"
Nov 01 03:13:47.284946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.284921    4892 scope.go:122] "RemoveContainer" containerID="163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"
Nov 01 03:13:47.285276 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:13:47.285257    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e\": container with ID starting with 163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e not found: ID does not exist" containerID="163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"
Nov 01 03:13:47.285326 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.285287    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e"} err="failed to get container status \"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e\": rpc error: code = NotFound desc = could not find container \"163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e\": container with ID starting with 163f57eed1fc88cff7956eb4cba8709131b91e25af6dcaa650b359dac0b7172e not found: ID does not exist"
Nov 01 03:13:47.454686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.454648    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:47.472185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.472155    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6d669c7fcd-zqlpr"]
Nov 01 03:13:47.699114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:47.699031    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" path="/var/lib/kubelet/pods/9935cbf0-95c6-412d-b2c7-e3fedd37e7a1/volumes"
Nov 01 03:13:56.184720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:56.184687    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-p8ncj"
Nov 01 03:13:57.186999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:13:57.186955    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:13:57.612741 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:13:57.612697    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:14:27.609448 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:14:27.609409    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:14:40.356730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.356691    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"]
Nov 01 03:14:40.357141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.357058    4892 cpu_manager.go:414] "containerMap: removing container" podUID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" containerName="maas-api"
Nov 01 03:14:40.357141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.357071    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" containerName="maas-api"
Nov 01 03:14:40.357141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.357130    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="9935cbf0-95c6-412d-b2c7-e3fedd37e7a1" containerName="maas-api"
Nov 01 03:14:40.359090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.359062    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.418993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.418954    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"]
Nov 01 03:14:40.445857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.445821    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4vj2\" (UniqueName: \"kubernetes.io/projected/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-kube-api-access-w4vj2\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.446052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.445865    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-cert\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.546591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.546546    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4vj2\" (UniqueName: \"kubernetes.io/projected/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-kube-api-access-w4vj2\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.546591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.546597    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-cert\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.549168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.549142    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-cert\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.608760 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.608681    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4vj2\" (UniqueName: \"kubernetes.io/projected/f1dd781d-10fb-44e4-b09b-2e6387aa2f5e-kube-api-access-w4vj2\") pod \"llmisvc-controller-manager-6466c554d8-wtt45\" (UID: \"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:40.671039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:40.670966    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:41.044644 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:41.044613    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"]
Nov 01 03:14:41.048050 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:14:41.048015    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1dd781d_10fb_44e4_b09b_2e6387aa2f5e.slice/crio-190c38971e34fe8574b47a36bd97484600c540a4863ea00c7142b4e865857244 WatchSource:0}: Error finding container 190c38971e34fe8574b47a36bd97484600c540a4863ea00c7142b4e865857244: Status 404 returned error can't find the container with id 190c38971e34fe8574b47a36bd97484600c540a4863ea00c7142b4e865857244
Nov 01 03:14:41.509415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:41.509378    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45" event={"ID":"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e","Type":"ContainerStarted","Data":"190c38971e34fe8574b47a36bd97484600c540a4863ea00c7142b4e865857244"}
Nov 01 03:14:42.515828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:42.515793    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45" event={"ID":"f1dd781d-10fb-44e4-b09b-2e6387aa2f5e","Type":"ContainerStarted","Data":"91c3f11d1426ac111ce958e56939b8d2ea6040105fdb8ccd8575b6fc5292f367"}
Nov 01 03:14:42.516281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:42.515930    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:14:42.633194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:14:42.633138    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45" podStartSLOduration=1.94242432 podStartE2EDuration="2.633122229s" totalImagesPullingTime="690.697909ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:14:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:14:42.631836286 +0000 UTC m=+449.047735815" watchObservedRunningTime="2026-11-01 03:14:42.633122229 +0000 UTC m=+449.049021759"
Nov 01 03:14:57.611380 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:14:57.611301    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:15:13.522287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:13.522253    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-wtt45"
Nov 01 03:15:13.690470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:13.690440    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:15:13.690705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:13.690672    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" podUID="59122762-6f85-4f60-9881-1c193087cfbe" containerName="manager" containerID="cri-o://f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee" gracePeriod=30
Nov 01 03:15:13.944442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:13.944418    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:15:14.038016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.037960    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\" (UniqueName: \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\") pod \"59122762-6f85-4f60-9881-1c193087cfbe\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") "
Nov 01 03:15:14.038016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.038019    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\" (UniqueName: \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\") pod \"59122762-6f85-4f60-9881-1c193087cfbe\" (UID: \"59122762-6f85-4f60-9881-1c193087cfbe\") "
Nov 01 03:15:14.040190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.040152    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert" pod "59122762-6f85-4f60-9881-1c193087cfbe" (UID: "59122762-6f85-4f60-9881-1c193087cfbe"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:15:14.040190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.040183    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf" pod "59122762-6f85-4f60-9881-1c193087cfbe" (UID: "59122762-6f85-4f60-9881-1c193087cfbe"). InnerVolumeSpecName "kube-api-access-9vhsf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:15:14.139551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.139455    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vhsf\" (UniqueName: \"kubernetes.io/projected/59122762-6f85-4f60-9881-1c193087cfbe-kube-api-access-9vhsf\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:15:14.139551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.139496    4892 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59122762-6f85-4f60-9881-1c193087cfbe-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:15:14.651194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.651161    4892 generic.go:356] "Generic (PLEG): container finished" podID="59122762-6f85-4f60-9881-1c193087cfbe" containerID="f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee" exitCode=0
Nov 01 03:15:14.651613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.651208    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"
Nov 01 03:15:14.651613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.651218    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" event={"ID":"59122762-6f85-4f60-9881-1c193087cfbe","Type":"ContainerDied","Data":"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"}
Nov 01 03:15:14.651613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.651286    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj" event={"ID":"59122762-6f85-4f60-9881-1c193087cfbe","Type":"ContainerDied","Data":"a3b89df35ec57a34d8e1cff803bf0973577b83ec0408307a157bfd5dc2717ec7"}
Nov 01 03:15:14.651613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.651309    4892 scope.go:122] "RemoveContainer" containerID="f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"
Nov 01 03:15:14.660943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.660918    4892 scope.go:122] "RemoveContainer" containerID="f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"
Nov 01 03:15:14.661244 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:15:14.661222    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee\": container with ID starting with f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee not found: ID does not exist" containerID="f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"
Nov 01 03:15:14.661300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.661253    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee"} err="failed to get container status \"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee\": rpc error: code = NotFound desc = could not find container \"f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee\": container with ID starting with f16181e62f7a5a785961944741a4b6ccf80322a4c96202cf0db4f9defdcfeeee not found: ID does not exist"
Nov 01 03:15:14.818249 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.818215    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:15:14.867166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:14.867133    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-n2hxj"]
Nov 01 03:15:15.699096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:15.699063    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="59122762-6f85-4f60-9881-1c193087cfbe" path="/var/lib/kubelet/pods/59122762-6f85-4f60-9881-1c193087cfbe/volumes"
Nov 01 03:15:19.681903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:19.681871    4892 generic.go:356] "Generic (PLEG): container finished" podID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerID="d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6" exitCode=1
Nov 01 03:15:19.682338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:19.681942    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerDied","Data":"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"}
Nov 01 03:15:19.682395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:19.682343    4892 scope.go:122] "RemoveContainer" containerID="d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"
Nov 01 03:15:20.688193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:20.688155    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerStarted","Data":"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"}
Nov 01 03:15:20.688582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:20.688373    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:15:27.610266 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:15:27.610230    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:15:31.699142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:15:31.699113    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:15:57.611080 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:15:57.611040    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:16:15.775989 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.775936    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:16:15.776427 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.776398    4892 cpu_manager.go:414] "containerMap: removing container" podUID="59122762-6f85-4f60-9881-1c193087cfbe" containerName="manager"
Nov 01 03:16:15.776427 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.776419    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="59122762-6f85-4f60-9881-1c193087cfbe" containerName="manager"
Nov 01 03:16:15.776510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.776495    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="59122762-6f85-4f60-9881-1c193087cfbe" containerName="manager"
Nov 01 03:16:15.779692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.779672    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.832088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.832056    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 01 03:16:15.866380 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866327    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866380 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866380    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866405    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866490    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s97wv\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866553    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866600    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866687    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866706    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.866838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.866788    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.883311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.883276    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:16:15.918910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.918870    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v7nmz\""
Nov 01 03:16:15.968205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968163    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968206    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968225    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968241    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968259    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s97wv\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968294    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968321    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968358    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968382    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968653    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968727    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968800    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.968931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.968865    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.969067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.969046    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.970666 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.970644    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:15.970900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:15.970880    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:16.029546 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:16.029465    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:16.029546 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:16.029501    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s97wv\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-hfnvd\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:16.092685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:16.092638    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:16.521697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:16.521664    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:16:16.524839 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:16:16.524811    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd59e7721_5189_4c9c_930c_c2d35258a032.slice/crio-db2a60ed6c5f9573152a0db3d689bc9bf5dad9051abfaaba979b6c5e45222cc2 WatchSource:0}: Error finding container db2a60ed6c5f9573152a0db3d689bc9bf5dad9051abfaaba979b6c5e45222cc2: Status 404 returned error can't find the container with id db2a60ed6c5f9573152a0db3d689bc9bf5dad9051abfaaba979b6c5e45222cc2
Nov 01 03:16:16.930876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:16.930788    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" event={"ID":"d59e7721-5189-4c9c-930c-c2d35258a032","Type":"ContainerStarted","Data":"db2a60ed6c5f9573152a0db3d689bc9bf5dad9051abfaaba979b6c5e45222cc2"}
Nov 01 03:16:20.502604 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:20.502573    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:20.502931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:20.502641    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:20.502931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:20.502667    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:20.951888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:20.951844    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" event={"ID":"d59e7721-5189-4c9c-930c-c2d35258a032","Type":"ContainerStarted","Data":"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"}
Nov 01 03:16:21.093370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:21.093337    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:21.094807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:21.094779    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:21.094913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:21.094829    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:21.129040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:21.128947    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podStartSLOduration=2.154018437 podStartE2EDuration="6.128927619s" totalImagesPullingTime="3.974909182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:16:21.127778623 +0000 UTC m=+547.543678153" watchObservedRunningTime="2026-11-01 03:16:21.128927619 +0000 UTC m=+547.544827149"
Nov 01 03:16:22.094050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:22.093930    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:22.094050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:22.094019    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:23.093237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:23.093203    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:23.093426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:23.093256    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:24.093971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:24.093934    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:24.094411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:24.094014    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:25.093106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:25.093064    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:25.093303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:25.093134    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:26.093571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:26.093531    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:26.094106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:26.093606    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:27.094190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:27.094143    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body=
Nov 01 03:16:27.094698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:27.094232    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused"
Nov 01 03:16:27.612920 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:16:27.612877    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:16:28.773115 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:28.773082    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:28.773691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:28.773390    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:28.774377 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:28.774360    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:16:30.891951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.891915    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:16:30.896595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.896559    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933561    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933611    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933692    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933741    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933808    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.933861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933842    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qdk7\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.934059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933902    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.934059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.933933    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:30.934059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:30.934017    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035279 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035232    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035318    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035342    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035365    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035386    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035426    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035463    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qdk7\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035498    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035525    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035690    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035893    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.035944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.035921    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.036092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.036043    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.036299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.036276    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.037864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.037842    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.038089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.038070    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.098339 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.098298    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.098830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.098805    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qdk7\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-qw5tz\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.110567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.110531    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:16:31.211713 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.211620    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:31.643586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.643548    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:16:31.646467 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:16:31.646429    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32aceb95_8220_4d76_b073_9df54c1906f6.slice/crio-12810474274dd40d2e5bc7a4e5073bdbaf4bd42a5ecba83f0f6456ab78eaa534 WatchSource:0}: Error finding container 12810474274dd40d2e5bc7a4e5073bdbaf4bd42a5ecba83f0f6456ab78eaa534: Status 404 returned error can't find the container with id 12810474274dd40d2e5bc7a4e5073bdbaf4bd42a5ecba83f0f6456ab78eaa534
Nov 01 03:16:31.649470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.649439    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:31.649550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.649506    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:31.649550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:31.649539    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:16:32.004564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:32.004525    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" event={"ID":"32aceb95-8220-4d76-b073-9df54c1906f6","Type":"ContainerStarted","Data":"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"}
Nov 01 03:16:32.005001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:32.004586    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" event={"ID":"32aceb95-8220-4d76-b073-9df54c1906f6","Type":"ContainerStarted","Data":"12810474274dd40d2e5bc7a4e5073bdbaf4bd42a5ecba83f0f6456ab78eaa534"}
Nov 01 03:16:32.212091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:32.212046    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:32.213759 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:32.213729    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:32.213922 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:32.213791    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:33.212540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:33.212499    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:33.212958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:33.212558    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:34.212506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:34.212460    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:34.212692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:34.212522    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:35.212616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:35.212575    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:35.212802 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:35.212643    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:36.212464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:36.212422    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:36.212662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:36.212493    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:37.212842 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:37.212802    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:37.213285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:37.212858    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:38.212890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:38.212851    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body=
Nov 01 03:16:38.213316 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:38.212911    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused"
Nov 01 03:16:39.992052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:39.992013    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:39.993145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:39.993118    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:39.994033 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:39.994013    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:16:40.212777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:16:40.212715    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podStartSLOduration=10.212699797 podStartE2EDuration="10.212699797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:16:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:16:32.127741521 +0000 UTC m=+558.543641051" watchObservedRunningTime="2026-11-01 03:16:40.212699797 +0000 UTC m=+566.628599326"
Nov 01 03:16:57.610699 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:16:57.610655    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:17:13.740732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:13.740703    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:17:13.741438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:13.741420    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:17:16.232639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.232607    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:17:16.239598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.239574    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.343413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.343380    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:17:16.358693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358663    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.358854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358753    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.358854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358781    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.358935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358859    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.358935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358879    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.358935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.358894    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.359086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.359009    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5d48\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.359086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.359032    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.359086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.359052    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.460361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.460324    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.460361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.460374    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.460616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.460417    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.460941    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.460527    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.461051    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.461147    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5d48\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.461176    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.461203    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.461363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.461245    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.462932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.462210    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.463309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.463284    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.463583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.463557    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.463733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.463705    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.469788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.469754    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.474786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.474757    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.521706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.521666    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.534838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.534807    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5d48\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-2w72r\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.552766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.552740    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:16.982571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.982544    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:17:16.985145 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:17:16.985121    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2a19921_1271_436a_ab90_26a79ff9e83c.slice/crio-34cec4185c12dd8a4626bc11223d98ae66b71b460687460781f8ce25cbcb60ec WatchSource:0}: Error finding container 34cec4185c12dd8a4626bc11223d98ae66b71b460687460781f8ce25cbcb60ec: Status 404 returned error can't find the container with id 34cec4185c12dd8a4626bc11223d98ae66b71b460687460781f8ce25cbcb60ec
Nov 01 03:17:16.987311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.987271    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:16.987423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.987352    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:16.987423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:16.987386    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:17.234539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:17.234455    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" event={"ID":"f2a19921-1271-436a-ab90-26a79ff9e83c","Type":"ContainerStarted","Data":"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"}
Nov 01 03:17:17.234539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:17.234510    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" event={"ID":"f2a19921-1271-436a-ab90-26a79ff9e83c","Type":"ContainerStarted","Data":"34cec4185c12dd8a4626bc11223d98ae66b71b460687460781f8ce25cbcb60ec"}
Nov 01 03:17:17.553288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:17.553245    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:18.554593 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:18.554550    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:18.554961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:18.554619    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:19.553299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:19.553266    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:19.553489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:19.553322    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:20.553231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:20.553196    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:20.553612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:20.553248    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:21.554090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:21.554051    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:21.554474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:21.554109    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:22.553777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:22.553737    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:22.553964 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:22.553795    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:23.553655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:23.553617    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Nov 01 03:17:23.554086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:23.553674    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Nov 01 03:17:25.217119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:25.217082    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:25.217590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:25.217458    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:25.218284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:25.218261    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:17:25.387899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:25.387842    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podStartSLOduration=9.387826972 podStartE2EDuration="9.387826972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:17:17.358311814 +0000 UTC m=+603.774211325" watchObservedRunningTime="2026-11-01 03:17:25.387826972 +0000 UTC m=+611.803726501"
Nov 01 03:17:27.609194 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:17:27.609151    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:17:37.060445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.060407    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:17:37.064193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.064173    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.116856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.116805    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-d7f535-openshift-default-dockercfg-7hqkk\""
Nov 01 03:17:37.153822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.153778    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbs82\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.153837    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.153916    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154014    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154105    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154146    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154233    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154264    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.154336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.154303    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.167606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.167577    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:17:37.255430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255387    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255445    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255472    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255501    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbs82\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255522    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255540    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255690    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255761    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.255892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255796    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.256137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.255929    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.256137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.256097    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.256389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.256365    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.256618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.256597    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.256719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.256700    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.258081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.258054    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.258262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.258240    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.328806 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.328726    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.328943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.328865    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbs82\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.377072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.377029    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:37.398143 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.398113    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:17:37.401423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.401400    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.458492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.454156    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-c22dd8-openshift-default-dockercfg-64gt8\""
Nov 01 03:17:37.466095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.466015    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:17:37.558969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.558931    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.558969    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559107    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559194    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559286    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559355    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559450    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559478    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.559541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.559530    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k52h\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.660840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660748    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.660840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660805    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660844    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660882    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k52h\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660936    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.660963    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661225    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661222    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661262    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661289    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661340    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661340    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661576 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661475    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.661851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.661827    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.663364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.663344    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.663446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.663422    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.724447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.724409    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.725102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.725076    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k52h\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:37.820235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.820199    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:17:37.823626 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:17:37.823588    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e570a1_365e_44f2_b04d_c8f1a3ab898c.slice/crio-f5b1ec1ac311c563723348f70172f2d7e15eec61746c7f14c8d6f01083455608 WatchSource:0}: Error finding container f5b1ec1ac311c563723348f70172f2d7e15eec61746c7f14c8d6f01083455608: Status 404 returned error can't find the container with id f5b1ec1ac311c563723348f70172f2d7e15eec61746c7f14c8d6f01083455608
Nov 01 03:17:37.825902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.825863    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:37.825966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.825951    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:37.826026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:37.826010    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:38.015367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.015325    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:38.338787 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.338688    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerStarted","Data":"6117ddfb846785ed33005921b28fe8656602ad531717150bccb65130d15278df"}
Nov 01 03:17:38.338787 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.338746    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerStarted","Data":"f5b1ec1ac311c563723348f70172f2d7e15eec61746c7f14c8d6f01083455608"}
Nov 01 03:17:38.378184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.378146    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:38.383054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.383025    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:38.470828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.470758    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podStartSLOduration=1.470739895 podStartE2EDuration="1.470739895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:17:38.467403352 +0000 UTC m=+624.883302894" watchObservedRunningTime="2026-11-01 03:17:38.470739895 +0000 UTC m=+624.886639427"
Nov 01 03:17:38.473081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.473049    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:17:38.523799 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:17:38.523765    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53019ee_51d2_41a2_9b5f_45b0a23e992e.slice/crio-77500ec6a6764d8e8fc08ca68abc3c8947e3b8fe85bc6fe45cb8fc898461a2d8 WatchSource:0}: Error finding container 77500ec6a6764d8e8fc08ca68abc3c8947e3b8fe85bc6fe45cb8fc898461a2d8: Status 404 returned error can't find the container with id 77500ec6a6764d8e8fc08ca68abc3c8947e3b8fe85bc6fe45cb8fc898461a2d8
Nov 01 03:17:38.526539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.526504    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:38.526645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.526568    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:38.526645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:38.526599    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:17:39.344355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:39.344315    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerStarted","Data":"20b52f6116b2257226916bfaf33f72d6d75c456683470e5fbcaa31657ac82666"}
Nov 01 03:17:39.344829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:39.344382    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerStarted","Data":"77500ec6a6764d8e8fc08ca68abc3c8947e3b8fe85bc6fe45cb8fc898461a2d8"}
Nov 01 03:17:39.344829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:39.344427    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:39.345526 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:39.345503    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:17:39.465875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:39.465816    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podStartSLOduration=2.465797529 podStartE2EDuration="2.465797529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:17:39.464548558 +0000 UTC m=+625.880448123" watchObservedRunningTime="2026-11-01 03:17:39.465797529 +0000 UTC m=+625.881697056"
Nov 01 03:17:40.015794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:40.015763    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:40.021306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:40.021279    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:40.348738 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:40.348663    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:40.349588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:17:40.349570    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:17:57.615203 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:17:57.615123    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:18:04.035805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.035775    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:18:04.039484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.039452    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.095043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.095011    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-e8c789\""
Nov 01 03:18:04.097561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.097518    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:18:04.111707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.111680    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpwwd\" (UniqueName: \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.111859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.111717    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.141399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.141367    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-e8c789-dockercfg-cq2f5\""
Nov 01 03:18:04.214065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.212899    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.214065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.213112    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpwwd\" (UniqueName: \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.215216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.214493    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.280074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.280037    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpwwd\" (UniqueName: \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\") pod \"payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.352860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.352766    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:04.731431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.731403    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:18:04.734106 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:04.734074    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81a157c_9621_452c_a566_465756cab80a.slice/crio-06225f8293ed50300b35d2238c895215817ef4860b7118d1bb0c3f722f8e1e6a WatchSource:0}: Error finding container 06225f8293ed50300b35d2238c895215817ef4860b7118d1bb0c3f722f8e1e6a: Status 404 returned error can't find the container with id 06225f8293ed50300b35d2238c895215817ef4860b7118d1bb0c3f722f8e1e6a
Nov 01 03:18:04.736085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:04.736061    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:18:05.468351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:05.468313    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" event={"ID":"e81a157c-9621-452c-a566-465756cab80a","Type":"ContainerStarted","Data":"06225f8293ed50300b35d2238c895215817ef4860b7118d1bb0c3f722f8e1e6a"}
Nov 01 03:18:06.528091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.528053    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:18:06.535376 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.535349    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.590044 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.590007    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:18:06.635348 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.635307    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm25c\" (UniqueName: \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.635511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.635390    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.637856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.637834    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-c8cb9a\""
Nov 01 03:18:06.638000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.637952    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-c8cb9a-dockercfg-zsq9k\""
Nov 01 03:18:06.661011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.660955    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:18:06.667913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.667882    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.722621 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.722588    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:18:06.736671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.736634    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2knvn\" (UniqueName: \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.736671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.736671    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.736903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.736820    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm25c\" (UniqueName: \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.736903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.736888    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.737659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.737635    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.798782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.798707    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm25c\" (UniqueName: \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\") pod \"payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.838047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.838013    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2knvn\" (UniqueName: \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.838242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.838062    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.838930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.838904    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.849960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.849930    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:06.902305 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.902260    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2knvn\" (UniqueName: \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\") pod \"payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:06.983938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:06.983897    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:07.246145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.245728    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:18:07.249071 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:07.249033    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708543c0_447c_493b_9026_926c02ce35e6.slice/crio-b5801a90436c00d8210912430ada16f2911727e8255b01887808e26e6d35eb5c WatchSource:0}: Error finding container b5801a90436c00d8210912430ada16f2911727e8255b01887808e26e6d35eb5c: Status 404 returned error can't find the container with id b5801a90436c00d8210912430ada16f2911727e8255b01887808e26e6d35eb5c
Nov 01 03:18:07.379439 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.379238    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:18:07.400290 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:07.400255    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc33c7329_1dcd_4c77_8752_8cd96900ebb2.slice/crio-bc3c03d23564b24eed78c1b5bbba6c96f0f7ca76e11cfc125966b1eaa75be378 WatchSource:0}: Error finding container bc3c03d23564b24eed78c1b5bbba6c96f0f7ca76e11cfc125966b1eaa75be378: Status 404 returned error can't find the container with id bc3c03d23564b24eed78c1b5bbba6c96f0f7ca76e11cfc125966b1eaa75be378
Nov 01 03:18:07.478679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.478650    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" event={"ID":"c33c7329-1dcd-4c77-8752-8cd96900ebb2","Type":"ContainerStarted","Data":"bc3c03d23564b24eed78c1b5bbba6c96f0f7ca76e11cfc125966b1eaa75be378"}
Nov 01 03:18:07.480130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.480098    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" event={"ID":"e81a157c-9621-452c-a566-465756cab80a","Type":"ContainerStarted","Data":"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"}
Nov 01 03:18:07.480259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.480163    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:07.481251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.481232    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" event={"ID":"708543c0-447c-493b-9026-926c02ce35e6","Type":"ContainerStarted","Data":"b5801a90436c00d8210912430ada16f2911727e8255b01887808e26e6d35eb5c"}
Nov 01 03:18:07.597911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:07.597859    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" podStartSLOduration=1.9250602749999999 podStartE2EDuration="4.597843964s" totalImagesPullingTime="2.672783689s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:07.596204378 +0000 UTC m=+654.012103936" watchObservedRunningTime="2026-11-01 03:18:07.597843964 +0000 UTC m=+654.013743493"
Nov 01 03:18:08.491024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.490965    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" event={"ID":"708543c0-447c-493b-9026-926c02ce35e6","Type":"ContainerStarted","Data":"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"}
Nov 01 03:18:08.491218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.491077    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:08.492689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.492655    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" event={"ID":"c33c7329-1dcd-4c77-8752-8cd96900ebb2","Type":"ContainerStarted","Data":"21f1dc8a5cd652df5a0986aa23fb1b38e98f6d3a8af2533ce5e41923ae7ac443"}
Nov 01 03:18:08.492832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.492805    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:08.608744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.608635    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" podStartSLOduration=1.784439575 podStartE2EDuration="2.608614766s" totalImagesPullingTime="824.175191ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:08.607240669 +0000 UTC m=+655.023140199" watchObservedRunningTime="2026-11-01 03:18:08.608614766 +0000 UTC m=+655.024514337"
Nov 01 03:18:08.724155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:08.724038    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" podStartSLOduration=1.804361572 podStartE2EDuration="2.724012356s" totalImagesPullingTime="919.650784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:08.723805629 +0000 UTC m=+655.139705199" watchObservedRunningTime="2026-11-01 03:18:08.724012356 +0000 UTC m=+655.139911935"
Nov 01 03:18:11.436224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.436182    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:18:11.439821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.439799    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.484851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.484808    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g6r5\" (UniqueName: \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.485062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.484969    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.492298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.492271    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-c22dd8-dockercfg-r9tjf\""
Nov 01 03:18:11.543007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.542169    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-c22dd8\""
Nov 01 03:18:11.544598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.544565    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:18:11.585702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.585645    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.585899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.585717    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g6r5\" (UniqueName: \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.586456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.586434    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.647706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.647673    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g6r5\" (UniqueName: \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\") pod \"payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:11.753119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:11.753087    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:12.132582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.132501    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:18:12.135528 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:12.135488    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee61b8a2_75e4_4482_a70a_f97e1a640b5f.slice/crio-747a7bdb15d2791b38a32fa5b70fbfdbc91e76081050709aad8de805dcc91e91 WatchSource:0}: Error finding container 747a7bdb15d2791b38a32fa5b70fbfdbc91e76081050709aad8de805dcc91e91: Status 404 returned error can't find the container with id 747a7bdb15d2791b38a32fa5b70fbfdbc91e76081050709aad8de805dcc91e91
Nov 01 03:18:12.513578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.513539    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" event={"ID":"ee61b8a2-75e4-4482-a70a-f97e1a640b5f","Type":"ContainerStarted","Data":"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"}
Nov 01 03:18:12.514092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.513630    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" event={"ID":"ee61b8a2-75e4-4482-a70a-f97e1a640b5f","Type":"ContainerStarted","Data":"747a7bdb15d2791b38a32fa5b70fbfdbc91e76081050709aad8de805dcc91e91"}
Nov 01 03:18:12.514092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.513669    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:12.632848 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.632781    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" podStartSLOduration=1.632767654 podStartE2EDuration="1.632767654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:12.630923691 +0000 UTC m=+659.046823219" watchObservedRunningTime="2026-11-01 03:18:12.632767654 +0000 UTC m=+659.048667181"
Nov 01 03:18:12.989643 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.989547    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:18:12.995404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:12.995363    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.047956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.047910    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-ded2d7-serving-cert\""
Nov 01 03:18:13.097285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.097252    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 01 03:18:13.097285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.097267    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zx9nh\""
Nov 01 03:18:13.099200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.099180    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:18:13.100855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.100837    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vjvj\" (UniqueName: \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.100927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.100874    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.100927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.100903    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.101136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.101116    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.202588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.202542    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.202809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.202597    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.202809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.202695    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.202809 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:13.202736    4892 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-serving-cert: secret "maas-api-e2e-worker-w0-ded2d7-serving-cert" not found
Nov 01 03:18:13.202809 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:13.202801    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls podName:632be7bb-4a1f-43d9-8017-b858ddca490b nodeName:}" failed. No retries permitted until 2026-11-01 03:18:13.702779945 +0000 UTC m=+660.118679472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls") pod "maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" (UID: "632be7bb-4a1f-43d9-8017-b858ddca490b") : secret "maas-api-e2e-worker-w0-ded2d7-serving-cert" not found
Nov 01 03:18:13.203062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.202817    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vjvj\" (UniqueName: \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.203358 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.203331    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.205148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.205123    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.266337 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.266286    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vjvj\" (UniqueName: \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.708447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.708364    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.710736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.710708    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") pod \"maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:13.960650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.960569    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zx9nh\""
Nov 01 03:18:13.969679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:13.969646    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:14.366324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:14.366277    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:18:14.369203 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:14.369175    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632be7bb_4a1f_43d9_8017_b858ddca490b.slice/crio-f1d62d564f3f1daead9610bad85262463a804438f57e5408d09db961ed4bfd16 WatchSource:0}: Error finding container f1d62d564f3f1daead9610bad85262463a804438f57e5408d09db961ed4bfd16: Status 404 returned error can't find the container with id f1d62d564f3f1daead9610bad85262463a804438f57e5408d09db961ed4bfd16
Nov 01 03:18:14.550026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:14.548435    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" event={"ID":"632be7bb-4a1f-43d9-8017-b858ddca490b","Type":"ContainerStarted","Data":"f1d62d564f3f1daead9610bad85262463a804438f57e5408d09db961ed4bfd16"}
Nov 01 03:18:15.555226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:15.555124    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" event={"ID":"632be7bb-4a1f-43d9-8017-b858ddca490b","Type":"ContainerStarted","Data":"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"}
Nov 01 03:18:15.555640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:15.555265    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:15.675682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:15.675633    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" podStartSLOduration=2.777057791 podStartE2EDuration="3.67562043s" totalImagesPullingTime="898.562639ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:15.673592062 +0000 UTC m=+662.089491588" watchObservedRunningTime="2026-11-01 03:18:15.67562043 +0000 UTC m=+662.091520018"
Nov 01 03:18:18.495930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:18.495903    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:18:19.501446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:19.501418    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:18:19.501960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:19.501483    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:18:21.566352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:21.566321    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:18:22.593231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.593195    4892 generic.go:356] "Generic (PLEG): container finished" podID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerID="20b52f6116b2257226916bfaf33f72d6d75c456683470e5fbcaa31657ac82666" exitCode=0
Nov 01 03:18:22.593668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.593255    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerDied","Data":"20b52f6116b2257226916bfaf33f72d6d75c456683470e5fbcaa31657ac82666"}
Nov 01 03:18:22.593668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.593611    4892 scope.go:122] "RemoveContainer" containerID="20b52f6116b2257226916bfaf33f72d6d75c456683470e5fbcaa31657ac82666"
Nov 01 03:18:22.594971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.594939    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:22.595069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.595023    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:22.595069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:22.595057    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:23.546006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:23.545150    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:18:23.600559 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:23.600527    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerStarted","Data":"b8deacb9e974e7885f0ceb6ae6cc2f7c2abfea4012462be7b2e14a170ba96362"}
Nov 01 03:18:24.015571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:24.015533    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:18:24.016719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:24.016684    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 01 03:18:24.016852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:24.016763    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 01 03:18:25.016369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:25.016333    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 01 03:18:25.016774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:25.016390    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 01 03:18:25.349342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:25.349246    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:18:26.015962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:26.015928    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 01 03:18:26.016154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:26.015996    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 01 03:18:27.015929 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:27.015892    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 01 03:18:27.016354 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:27.015949    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 01 03:18:27.610636 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:27.610589    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:18:28.015962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.015928    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Nov 01 03:18:28.016390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.016037    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Nov 01 03:18:28.413772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.413692    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:18:28.417115 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.417091    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.473671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.473641    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-c8cb9a-serving-cert\""
Nov 01 03:18:28.502305 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.502239    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.502500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.502383    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.502500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.502432    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prlmz\" (UniqueName: \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.502633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.502590    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.524939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.524902    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:18:28.603580 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.603541    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.603792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.603600    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.603792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.603632    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.603792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.603654    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prlmz\" (UniqueName: \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.603792 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:28.603767    4892 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-serving-cert: secret "maas-api-e2e-worker-w2-c8cb9a-serving-cert" not found
Nov 01 03:18:28.604056 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:28.603843    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls podName:5f5b11c6-9f5f-4249-b723-3e0026576bf1 nodeName:}" failed. No retries permitted until 2026-11-01 03:18:29.103823667 +0000 UTC m=+675.519723189 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls") pod "maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" (UID: "5f5b11c6-9f5f-4249-b723-3e0026576bf1") : secret "maas-api-e2e-worker-w2-c8cb9a-serving-cert" not found
Nov 01 03:18:28.604476 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.604448    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.606157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.606132    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:28.668449 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:28.668342    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prlmz\" (UniqueName: \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:29.020401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.020370    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:18:29.021495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.021475    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:18:29.108620 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.108580    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:29.110973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.110939    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") pod \"maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:29.348034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.347934    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:29.735311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:29.735288    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:18:29.737733 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:18:29.737698    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5b11c6_9f5f_4249_b723_3e0026576bf1.slice/crio-87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f WatchSource:0}: Error finding container 87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f: Status 404 returned error can't find the container with id 87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f
Nov 01 03:18:30.655271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:30.655233    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" event={"ID":"5f5b11c6-9f5f-4249-b723-3e0026576bf1","Type":"ContainerStarted","Data":"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"}
Nov 01 03:18:30.655701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:30.655285    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" event={"ID":"5f5b11c6-9f5f-4249-b723-3e0026576bf1","Type":"ContainerStarted","Data":"87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f"}
Nov 01 03:18:30.655701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:30.655367    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:30.800091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:30.799992    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" podStartSLOduration=2.018776873 podStartE2EDuration="2.799962865s" totalImagesPullingTime="781.185992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:30.798860864 +0000 UTC m=+677.214760399" watchObservedRunningTime="2026-11-01 03:18:30.799962865 +0000 UTC m=+677.215862393"
Nov 01 03:18:36.665218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:36.665187    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:18:44.729076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.729041    4892 generic.go:356] "Generic (PLEG): container finished" podID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerID="6117ddfb846785ed33005921b28fe8656602ad531717150bccb65130d15278df" exitCode=0
Nov 01 03:18:44.729513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.729110    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerDied","Data":"6117ddfb846785ed33005921b28fe8656602ad531717150bccb65130d15278df"}
Nov 01 03:18:44.729569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.729516    4892 scope.go:122] "RemoveContainer" containerID="6117ddfb846785ed33005921b28fe8656602ad531717150bccb65130d15278df"
Nov 01 03:18:44.730811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.730787    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:44.730869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.730855    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:44.730905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:44.730894    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:18:45.751330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:45.751292    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerStarted","Data":"462397bdc0070e5e4ea8d8e4a4e6bbea95532b367bb82f262fc93017a826192f"}
Nov 01 03:18:46.377532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:46.377495    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:18:46.377765 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:46.377731    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 01 03:18:46.377817 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:46.377797    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 01 03:18:47.377482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:47.377446    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 01 03:18:47.377900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:47.377503    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 01 03:18:48.378307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:48.378266    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 01 03:18:48.378728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:48.378335    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 01 03:18:49.378134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.378098    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 01 03:18:49.378325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.378157    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 01 03:18:49.401103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.401068    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:18:49.403503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.403482    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.486158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.486121    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"default-dockercfg-hx678\""
Nov 01 03:18:49.486158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.486121    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"premium-w1-b12199-kserve-self-signed-certs\""
Nov 01 03:18:49.505714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.505676    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"openshift-service-ca.crt\""
Nov 01 03:18:49.505903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.505726    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"kube-root-ca.crt\""
Nov 01 03:18:49.507687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.507664    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:18:49.557894 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.557853    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.558123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.557917    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.558123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.557953    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgsw6\" (UniqueName: \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.558123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.558084    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.558301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.558229    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.558350 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.558318    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659566    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659626    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659662    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659690    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgsw6\" (UniqueName: \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659745    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.659952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.659810    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.660141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.660080    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.660386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.660363    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.661024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.660911    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.663046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.662961    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.663227 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.663208    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:49.725773 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:49.725698    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgsw6\" (UniqueName: \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\") pod \"premium-w1-b12199-kserve-85d78c6fbc-qbmvm\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") " pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:50.014552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:50.014513    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:50.377843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:50.377751    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Nov 01 03:18:50.377843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:50.377811    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Nov 01 03:18:50.417843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:50.417806    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:18:50.777898 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:50.777860    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" event={"ID":"34b8a7c8-e317-46f2-9630-e42b59e8edd0","Type":"ContainerStarted","Data":"01823214061ab632670cea3d82997ff10be06cf489444512365276801749a800"}
Nov 01 03:18:51.384015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:51.383966    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:18:51.384418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:51.384388    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:18:51.389540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:51.389518    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:18:52.790159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:52.790122    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" event={"ID":"34b8a7c8-e317-46f2-9630-e42b59e8edd0","Type":"ContainerStarted","Data":"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"}
Nov 01 03:18:52.908479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:52.908427    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podStartSLOduration=2.268914116 podStartE2EDuration="3.908411061s" totalImagesPullingTime="1.639496945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:18:52.907146903 +0000 UTC m=+699.323046432" watchObservedRunningTime="2026-11-01 03:18:52.908411061 +0000 UTC m=+699.324310589"
Nov 01 03:18:53.015556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:18:53.015518    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:18:57.610827 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:18:57.610790    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:19:00.015193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:00.015159    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:19:00.029707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:00.029674    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:19:00.842778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:00.842743    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:19:21.997111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:21.997024    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:19:22.003096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.003067    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.086815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.086780    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-b12199-openshift-default-dockercfg-lc8zw\""
Nov 01 03:19:22.094342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094306    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094363    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094499    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094557    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094653    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094684    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2t4x\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094746    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094804    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.094852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.094840    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.108126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.108087    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:19:22.195516 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195483    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195524    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195565    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195598    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195624    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195659    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.195720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195701    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195728    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2t4x\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.195763    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.196109    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.196135    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196562 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.196528    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196664 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.196538    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.196746 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.196720    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.197917 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.197896    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.198231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.198214    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.260260 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.260229    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.260480 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.260459    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2t4x\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.317241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.317201    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:22.757138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.757107    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:19:22.760042 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:19:22.760006    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cfb9067_0f82_439f_809d_35156a1542e3.slice/crio-ccc35c8000bfec3aa0aacf43972c47d0ed839dfb60bd9b9762ed1e808d6b2f36 WatchSource:0}: Error finding container ccc35c8000bfec3aa0aacf43972c47d0ed839dfb60bd9b9762ed1e808d6b2f36: Status 404 returned error can't find the container with id ccc35c8000bfec3aa0aacf43972c47d0ed839dfb60bd9b9762ed1e808d6b2f36
Nov 01 03:19:22.762339 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.762306    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:19:22.762502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.762387    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:19:22.762502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.762434    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:19:22.944730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.944693    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" event={"ID":"6cfb9067-0f82-439f-809d-35156a1542e3","Type":"ContainerStarted","Data":"0423180833d8e09e0f835eca3072ae71782e2f91d07608aae4c56093ddc3938a"}
Nov 01 03:19:22.944913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:22.944763    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" event={"ID":"6cfb9067-0f82-439f-809d-35156a1542e3","Type":"ContainerStarted","Data":"ccc35c8000bfec3aa0aacf43972c47d0ed839dfb60bd9b9762ed1e808d6b2f36"}
Nov 01 03:19:23.074352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:23.074220    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podStartSLOduration=2.074200951 podStartE2EDuration="2.074200951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:19:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:19:23.073287424 +0000 UTC m=+729.489186954" watchObservedRunningTime="2026-11-01 03:19:23.074200951 +0000 UTC m=+729.490100481"
Nov 01 03:19:23.318361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:23.318316    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:23.320094 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:23.320063    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:23.320219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:23.320121    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:24.317817 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:24.317780    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:24.318369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:24.317854    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:25.318089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:25.318057    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:25.318465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:25.318122    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:26.318543 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:26.318509    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:26.318948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:26.318562    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:27.317769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:27.317733    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:27.317973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:27.317800    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:27.610348 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:19:27.610256    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:19:28.318025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:28.317964    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Nov 01 03:19:28.318250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:28.318042    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Nov 01 03:19:29.537853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:29.537813    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:29.538398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:29.538379    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:29.539230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:29.539207    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:19:42.867955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.867921    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:19:42.871687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.871664    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912102    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912162    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912304    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912354    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912447    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4d4n\" (UniqueName: \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.912563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.912541    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:42.973399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.973361    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"simulator-w3-d7f535-kserve-self-signed-certs\""
Nov 01 03:19:42.973652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.973477    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"default-dockercfg-gw8jb\""
Nov 01 03:19:42.973652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.973536    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"openshift-service-ca.crt\""
Nov 01 03:19:42.973652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.973581    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"kube-root-ca.crt\""
Nov 01 03:19:42.975619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:42.975598    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:19:43.013798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.013759    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.013952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.013812    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.013952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.013871    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4d4n\" (UniqueName: \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.013952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.013915    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.014122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.014008    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.014122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.014046    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.014297 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.014264    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.014621 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.014379    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.014621 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.014583    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.016802 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.016772    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.017068 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.017048    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.075333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.075298    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4d4n\" (UniqueName: \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\") pod \"simulator-w3-d7f535-kserve-69ddfdfd94-67mrw\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") " pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.184420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.184334    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:43.566612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:43.566584    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:19:43.569068 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:19:43.569029    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70d0113f_df2b_48dc_91d7_821726cc5721.slice/crio-1babc2a0acf1921d8bd66deedf67933ef67283ec536f83d6a42ac78aef430585 WatchSource:0}: Error finding container 1babc2a0acf1921d8bd66deedf67933ef67283ec536f83d6a42ac78aef430585: Status 404 returned error can't find the container with id 1babc2a0acf1921d8bd66deedf67933ef67283ec536f83d6a42ac78aef430585
Nov 01 03:19:44.069063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:44.069029    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" event={"ID":"70d0113f-df2b-48dc-91d7-821726cc5721","Type":"ContainerStarted","Data":"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"}
Nov 01 03:19:44.069447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:44.069071    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" event={"ID":"70d0113f-df2b-48dc-91d7-821726cc5721","Type":"ContainerStarted","Data":"1babc2a0acf1921d8bd66deedf67933ef67283ec536f83d6a42ac78aef430585"}
Nov 01 03:19:44.184696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:44.184662    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:44.187041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:44.186995    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podStartSLOduration=2.186969611 podStartE2EDuration="2.186969611s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:19:44.185132589 +0000 UTC m=+750.601032137" watchObservedRunningTime="2026-11-01 03:19:44.186969611 +0000 UTC m=+750.602869139"
Nov 01 03:19:53.184797 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:53.184763    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:53.197097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:53.197067    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:54.127335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.127307    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:19:54.728265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.728224    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:19:54.733727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.733695    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.789920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.789881    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e8c789\"/\"openshift-service-ca.crt\""
Nov 01 03:19:54.792472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.792438    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:19:54.836430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836397    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e8c789\"/\"default-dockercfg-5fgpq\""
Nov 01 03:19:54.836630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836465    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-e8c789\"/\"kube-root-ca.crt\""
Nov 01 03:19:54.836630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836427    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e8c789\"/\"simulator-w4-e8c789-kserve-self-signed-certs\""
Nov 01 03:19:54.836630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836579    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.836630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836621    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.836854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836642    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.836854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836696    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nkcr\" (UniqueName: \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.836854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836750    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.837039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.836922    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938460    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938586    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938622    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938649    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938841 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938681    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nkcr\" (UniqueName: \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.938841 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938737    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.939009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.938955    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.939202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.939058    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.939202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.939097    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.941109 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.941082    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:54.941228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:54.941211    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:55.010719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:55.010678    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nkcr\" (UniqueName: \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\") pod \"simulator-w4-e8c789-kserve-ccc898c4c-vm69k\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") " pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:55.047889 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:55.047847    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:55.425007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:55.424969    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:19:55.427649 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:19:55.427620    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod336471d8_7aed_45c6_9695_1231ecdf5eb6.slice/crio-96eca9621c845eaabac0792a8c67fe68bc37145f3eddf75c30f4864ff18334b2 WatchSource:0}: Error finding container 96eca9621c845eaabac0792a8c67fe68bc37145f3eddf75c30f4864ff18334b2: Status 404 returned error can't find the container with id 96eca9621c845eaabac0792a8c67fe68bc37145f3eddf75c30f4864ff18334b2
Nov 01 03:19:56.130056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:56.130023    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" event={"ID":"336471d8-7aed-45c6-9695-1231ecdf5eb6","Type":"ContainerStarted","Data":"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"}
Nov 01 03:19:56.130464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:56.130069    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" event={"ID":"336471d8-7aed-45c6-9695-1231ecdf5eb6","Type":"ContainerStarted","Data":"96eca9621c845eaabac0792a8c67fe68bc37145f3eddf75c30f4864ff18334b2"}
Nov 01 03:19:56.248218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:56.248168    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podStartSLOduration=2.248152721 podStartE2EDuration="2.248152721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:19:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:19:56.246547381 +0000 UTC m=+762.662446912" watchObservedRunningTime="2026-11-01 03:19:56.248152721 +0000 UTC m=+762.664052250"
Nov 01 03:19:57.048545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:19:57.048497    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:19:57.610522 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:19:57.610486    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:20:05.049041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:05.049000    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:20:05.061280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:05.061250    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:20:05.184394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:05.184364    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:20:27.610802 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:20:27.610764    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:20:48.605729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.605694    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:20:48.609476 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.609459    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675158    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675293    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675401    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q6wc\" (UniqueName: \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675471    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675583    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.675663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.675646    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.711828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.711797    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-c22dd8\"/\"premium-w5-c22dd8-kserve-self-signed-certs\""
Nov 01 03:20:48.711828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.711818    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-c22dd8\"/\"openshift-service-ca.crt\""
Nov 01 03:20:48.711828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.711818    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-c22dd8\"/\"default-dockercfg-khjq4\""
Nov 01 03:20:48.712060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.711885    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-c22dd8\"/\"kube-root-ca.crt\""
Nov 01 03:20:48.713929 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.713907    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:20:48.776810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.776778    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.776818    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q6wc\" (UniqueName: \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.776846    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.776899    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.776925    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.777248    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.777362    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.777408    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.777597 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.777575    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.779261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.779226    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.779419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.779401    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.838635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.838592    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q6wc\" (UniqueName: \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\") pod \"premium-w5-c22dd8-kserve-7967d66f6d-zbmmj\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") " pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:48.946386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:48.946287    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:49.362071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:49.362043    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:20:49.364851 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:20:49.364812    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e6434e1_0f34_4169_ae1f_5efd29007ce0.slice/crio-74c796d9f5f7f628bef3b98301568dca394f7f52b5a2cda330aa126bdfc9f28a WatchSource:0}: Error finding container 74c796d9f5f7f628bef3b98301568dca394f7f52b5a2cda330aa126bdfc9f28a: Status 404 returned error can't find the container with id 74c796d9f5f7f628bef3b98301568dca394f7f52b5a2cda330aa126bdfc9f28a
Nov 01 03:20:49.409328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:49.409298    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" event={"ID":"5e6434e1-0f34-4169-ae1f-5efd29007ce0","Type":"ContainerStarted","Data":"74c796d9f5f7f628bef3b98301568dca394f7f52b5a2cda330aa126bdfc9f28a"}
Nov 01 03:20:50.415859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:50.415824    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" event={"ID":"5e6434e1-0f34-4169-ae1f-5efd29007ce0","Type":"ContainerStarted","Data":"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"}
Nov 01 03:20:50.534166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:50.534109    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podStartSLOduration=2.534091613 podStartE2EDuration="2.534091613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:20:50.533658235 +0000 UTC m=+816.949557766" watchObservedRunningTime="2026-11-01 03:20:50.534091613 +0000 UTC m=+816.949991141"
Nov 01 03:20:50.947355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:50.947319    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:57.611513 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:20:57.611426    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:20:58.946669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:58.946631    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:58.959785 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:58.959752    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:59.166916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.166884    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:20:59.170587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.170563    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.229424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.229330    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"premium-w3-d7f535-kserve-self-signed-certs\""
Nov 01 03:20:59.232520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.232491    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:20:59.279949 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.279914    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.280124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.280083    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.280167 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.280138    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.280203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.280179    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.280295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.280272    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8bl9\" (UniqueName: \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.280334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.280308    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381349    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381423    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381461    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381500    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8bl9\" (UniqueName: \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381527    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381578    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.381943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.381916    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.382046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.382009    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.382046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.382021    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.383816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.383789    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.384221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.384199    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.444140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.444106    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8bl9\" (UniqueName: \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\") pod \"premium-w3-d7f535-kserve-6f5cd7f459-wv8vz\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") " pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.474569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.474541    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:20:59.483341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.483282    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:20:59.883281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:20:59.883250    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:20:59.885573 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:20:59.885546    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02dcb23a_9c75_4df2_befa_09f4b87607a5.slice/crio-93063057c6d646b0b25f4aed00bc68d37ea60c57a6e85d10f76ec8e0a2d3f816 WatchSource:0}: Error finding container 93063057c6d646b0b25f4aed00bc68d37ea60c57a6e85d10f76ec8e0a2d3f816: Status 404 returned error can't find the container with id 93063057c6d646b0b25f4aed00bc68d37ea60c57a6e85d10f76ec8e0a2d3f816
Nov 01 03:21:00.471708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:00.471668    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" event={"ID":"02dcb23a-9c75-4df2-befa-09f4b87607a5","Type":"ContainerStarted","Data":"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"}
Nov 01 03:21:00.472307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:00.471719    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" event={"ID":"02dcb23a-9c75-4df2-befa-09f4b87607a5","Type":"ContainerStarted","Data":"93063057c6d646b0b25f4aed00bc68d37ea60c57a6e85d10f76ec8e0a2d3f816"}
Nov 01 03:21:00.484201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:00.484165    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:21:00.589801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:00.589737    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podStartSLOduration=1.589719189 podStartE2EDuration="1.589719189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:20:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:21:00.588511266 +0000 UTC m=+827.004410797" watchObservedRunningTime="2026-11-01 03:21:00.589719189 +0000 UTC m=+827.005618717"
Nov 01 03:21:09.484488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:09.484446    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:21:09.500554 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:09.500523    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:21:09.536789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:09.536754    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:21:23.135818 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.135777    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:21:23.140884 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.140856    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.235672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.235634    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds7dh\" (UniqueName: \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.235870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.235722    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.235870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.235809    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.235966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.235923    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.236028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.235996    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.236067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.236035    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.246502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.246468    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c8cb9a\"/\"openshift-service-ca.crt\""
Nov 01 03:21:23.246502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.246484    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-c8cb9a\"/\"kube-root-ca.crt\""
Nov 01 03:21:23.246710 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.246613    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c8cb9a\"/\"premium-w2-c8cb9a-kserve-self-signed-certs\""
Nov 01 03:21:23.246710 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.246690    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-c8cb9a\"/\"default-dockercfg-dxgc9\""
Nov 01 03:21:23.248513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.248493    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:21:23.336868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.336820    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.336868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.336860    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.336906    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.336926    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.336945    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.337060    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds7dh\" (UniqueName: \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.337330    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.337386    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.337507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.337406    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.339349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.339326    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.339748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.339725    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.400691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.400605    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds7dh\" (UniqueName: \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\") pod \"premium-w2-c8cb9a-kserve-d59774988-7n78z\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") " pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.454856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.454816    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:23.861611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:23.861581    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:21:23.864121 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:21:23.864091    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode46a4ca9_5b9e_4b28_b3c5_0b7a05db2981.slice/crio-b2cb1d4ea1aa3dec9f6fac7e646c8373b8b33671a4285c4c5f982ff14c128e79 WatchSource:0}: Error finding container b2cb1d4ea1aa3dec9f6fac7e646c8373b8b33671a4285c4c5f982ff14c128e79: Status 404 returned error can't find the container with id b2cb1d4ea1aa3dec9f6fac7e646c8373b8b33671a4285c4c5f982ff14c128e79
Nov 01 03:21:24.607561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:24.607524    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" event={"ID":"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981","Type":"ContainerStarted","Data":"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"}
Nov 01 03:21:24.607561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:24.607566    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" event={"ID":"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981","Type":"ContainerStarted","Data":"b2cb1d4ea1aa3dec9f6fac7e646c8373b8b33671a4285c4c5f982ff14c128e79"}
Nov 01 03:21:24.724143 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:24.724071    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podStartSLOduration=1.724055036 podStartE2EDuration="1.724055036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:21:24.723694967 +0000 UTC m=+851.139594490" watchObservedRunningTime="2026-11-01 03:21:24.724055036 +0000 UTC m=+851.139954562"
Nov 01 03:21:25.455109 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:25.455072    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:27.610931 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:21:27.610895    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:21:33.455156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:33.455111    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:33.468170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:33.468138    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:33.668039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:33.668010    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:21:54.858212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.858174    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:21:54.862015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.861994    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.914577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.914550    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-c8cb9a-openshift-default-dockercfg-r4cfc\""
Nov 01 03:21:54.971635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.971594    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:21:54.972681 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.972649    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.972890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.972869    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973049    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973249    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973376    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973483    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkm8b\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973672    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973786    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:54.973899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:54.973884    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075191 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075151    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075204    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075331    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkm8b\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075414    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075449    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075476    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075553    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075581    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075589    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075621    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075633    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.075948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075867    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.076033 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.075968    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.076262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.076242    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.078065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.078030    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.078065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.078054    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.137164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.137092    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.137381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.137356    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkm8b\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.176358 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.176312    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:55.614324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.614196    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:21:55.616539 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:21:55.616506    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod385d226c_e36e_4ac1_a508_f651fde5e2fe.slice/crio-0ac6d8c445087c1dea93d6d56993081e9bec8d09312fe6f4308c86bce5d75441 WatchSource:0}: Error finding container 0ac6d8c445087c1dea93d6d56993081e9bec8d09312fe6f4308c86bce5d75441: Status 404 returned error can't find the container with id 0ac6d8c445087c1dea93d6d56993081e9bec8d09312fe6f4308c86bce5d75441
Nov 01 03:21:55.618941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.618907    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:21:55.619071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.619004    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:21:55.619071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.619050    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:21:55.768355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.768319    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" event={"ID":"385d226c-e36e-4ac1-a508-f651fde5e2fe","Type":"ContainerStarted","Data":"ab887c4b9b131d3f56246dc222ba77cbf6bc1c37baf56357326d4cb0801eceeb"}
Nov 01 03:21:55.768525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.768377    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" event={"ID":"385d226c-e36e-4ac1-a508-f651fde5e2fe","Type":"ContainerStarted","Data":"0ac6d8c445087c1dea93d6d56993081e9bec8d09312fe6f4308c86bce5d75441"}
Nov 01 03:21:55.890599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:55.890489    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podStartSLOduration=1.890474072 podStartE2EDuration="1.890474072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:21:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:21:55.888197254 +0000 UTC m=+882.304096790" watchObservedRunningTime="2026-11-01 03:21:55.890474072 +0000 UTC m=+882.306373600"
Nov 01 03:21:56.177005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:56.176891    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:21:56.178611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:56.178583    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:21:56.178719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:56.178631    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:21:57.177095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:57.177052    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:21:57.177586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:57.177113    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:21:57.611659 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:21:57.611610    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:21:58.177619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:58.177582    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:21:58.178053 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:58.177640    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:21:59.177515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:59.177481    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:21:59.177673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:21:59.177540    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:22:00.176844 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:00.176808    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:22:00.177034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:00.176867    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:22:01.176808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:01.176764    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Nov 01 03:22:01.177249 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:01.176831    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Nov 01 03:22:02.375199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:02.375168    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:22:02.375852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:02.375827    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:22:02.377402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:02.377376    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:22:13.776213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:13.776188    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:22:13.778438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:13.778419    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:22:26.271063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:26.270110    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-2w72r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:22:26.271063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:26.270218    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:22:27.610752 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:22:27.610717    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:22:53.671829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.671787    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:22:53.684709 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.684669    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.782628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.781952    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:22:53.798750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.798718    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.798919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.798771    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.798919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.798798    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.798937    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.798974    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.799014    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.799053    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.799096    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.799139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.799111    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnd4h\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900378    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900425    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900443    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900470    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900492    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900511    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900556    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900587    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.900612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900608    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnd4h\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.901063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900853    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.901063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.900904    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.901154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.901132    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.901205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.901140    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.901330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.901312    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.903390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.903362    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.903525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.903508    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.967933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.967850    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:53.968139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:53.968006    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnd4h\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:54.000955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:54.000930    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:54.475114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:54.475078    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:22:54.478516 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:22:54.478484    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9fd8137_b090_4280_b60b_ed4416d09cef.slice/crio-67a1e913294f57578d102a8985671949337ad9558b9f7a1b234543934fd073c5 WatchSource:0}: Error finding container 67a1e913294f57578d102a8985671949337ad9558b9f7a1b234543934fd073c5: Status 404 returned error can't find the container with id 67a1e913294f57578d102a8985671949337ad9558b9f7a1b234543934fd073c5
Nov 01 03:22:54.480798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:54.480768    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:22:54.480870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:54.480832    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:22:54.480870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:54.480857    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:22:55.109532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:55.109492    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" event={"ID":"e9fd8137-b090-4280-b60b-ed4416d09cef","Type":"ContainerStarted","Data":"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"}
Nov 01 03:22:55.109915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:55.109558    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" event={"ID":"e9fd8137-b090-4280-b60b-ed4416d09cef","Type":"ContainerStarted","Data":"67a1e913294f57578d102a8985671949337ad9558b9f7a1b234543934fd073c5"}
Nov 01 03:22:56.001808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:56.001772    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:22:56.003165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:56.003143    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 01 03:22:56.003315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:56.003187    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 01 03:22:57.001653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:57.001614    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 01 03:22:57.002065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:57.001673    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 01 03:22:57.610593 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:22:57.610558    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:22:58.001814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:58.001734    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 01 03:22:58.001814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:58.001786    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 01 03:22:59.001889 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:59.001853    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 01 03:22:59.002302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:22:59.001916    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 01 03:23:00.001433 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:00.001394    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Nov 01 03:23:00.001616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:00.001447    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Nov 01 03:23:01.248783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:01.248749    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:23:01.249392 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:01.249368    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:23:01.250263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:01.250231    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:23:01.419769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:01.419712    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podStartSLOduration=8.419698036 podStartE2EDuration="8.419698036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:22:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:22:55.247644827 +0000 UTC m=+941.663544372" watchObservedRunningTime="2026-11-01 03:23:01.419698036 +0000 UTC m=+947.835597638"
Nov 01 03:23:24.543922 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.543883    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:23:24.547820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.547799    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.603161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.603124    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-ded2d7-openshift-default-dockercfg-glqxh\""
Nov 01 03:23:24.655039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.655004    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:23:24.664188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664153    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664256    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664308    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664345    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664418    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664443    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664470    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9mtr\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664575    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.664686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.664609    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765540    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765604    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765648    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765690    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765729    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.765799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765800    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765825    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765850    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9mtr\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.765952    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.766216    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.766389    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.766428    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.766590    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.766752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.766635    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.768157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.768134    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.768458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.768428    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.829872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.829805    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.830110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.830086    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9mtr\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:24.861400 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:24.861365    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:25.295247 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:25.295219    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:23:25.297876 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:23:25.297849    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4a3948_8dab_468a_8941_9bfd4644c2bb.slice/crio-0f235296bf4d8f4c39b3428d3e87a4e10e9553d7e5cdc9ed8ee1cca5e0b5adb9 WatchSource:0}: Error finding container 0f235296bf4d8f4c39b3428d3e87a4e10e9553d7e5cdc9ed8ee1cca5e0b5adb9: Status 404 returned error can't find the container with id 0f235296bf4d8f4c39b3428d3e87a4e10e9553d7e5cdc9ed8ee1cca5e0b5adb9
Nov 01 03:23:25.300217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:25.300184    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:23:25.300291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:25.300248    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:23:25.300291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:25.300278    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:23:26.301022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.300960    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" event={"ID":"1b4a3948-8dab-468a-8941-9bfd4644c2bb","Type":"ContainerStarted","Data":"8d97be2c36b9a294e96bd3f3e0b20bca232cae6404806e63953c7fac12d45746"}
Nov 01 03:23:26.301408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.301065    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" event={"ID":"1b4a3948-8dab-468a-8941-9bfd4644c2bb","Type":"ContainerStarted","Data":"0f235296bf4d8f4c39b3428d3e87a4e10e9553d7e5cdc9ed8ee1cca5e0b5adb9"}
Nov 01 03:23:26.471805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.471748    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podStartSLOduration=2.471734358 podStartE2EDuration="2.471734358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:23:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:23:26.470504635 +0000 UTC m=+972.886404156" watchObservedRunningTime="2026-11-01 03:23:26.471734358 +0000 UTC m=+972.887633885"
Nov 01 03:23:26.862056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.862015    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:26.863932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.863905    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 01 03:23:26.864080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:26.863950    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 01 03:23:27.613045 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:23:27.613008    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:23:27.862331 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:27.862277    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 01 03:23:27.862532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:27.862359    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 01 03:23:28.862732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:28.862693    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 01 03:23:28.863165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:28.862748    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 01 03:23:29.862274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:29.862230    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 01 03:23:29.862484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:29.862314    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 01 03:23:30.862360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:30.862315    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Nov 01 03:23:30.862866 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:30.862396    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Nov 01 03:23:32.182536 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:32.182491    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:32.183039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:32.183000    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:32.184114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:23:32.184092    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:23:57.610582 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:23:57.610492    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:24:12.698673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.698640    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:24:12.702443 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.702418    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781236 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781194    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781326    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781372    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zdf9\" (UniqueName: \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781481    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781512    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.781669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.781598    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.804450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.804421    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-e8c789\"/\"distinct-w4-e8c789-kserve-self-signed-certs\""
Nov 01 03:24:12.806684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.806662    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:24:12.882839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.882806    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.882868    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.882896    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zdf9\" (UniqueName: \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.882959    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.883031    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.883160    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.883411    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.883420    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.883574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.883513    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.885420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.885383    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.885675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.885659    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:12.944845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:12.944809    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zdf9\" (UniqueName: \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\") pod \"distinct-w4-e8c789-kserve-8fcb884b5-8hfsp\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") " pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:13.015241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:13.015201    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:13.413461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:13.413363    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:24:13.416105 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:24:13.416074    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1e2b14e7_6838_4c36_95c6_38526a72758d.slice/crio-33fdfc2ad0697150a7d5a2e5f596c85dc4c3b566ba272e7cfb23650d5f1b2863 WatchSource:0}: Error finding container 33fdfc2ad0697150a7d5a2e5f596c85dc4c3b566ba272e7cfb23650d5f1b2863: Status 404 returned error can't find the container with id 33fdfc2ad0697150a7d5a2e5f596c85dc4c3b566ba272e7cfb23650d5f1b2863
Nov 01 03:24:13.564043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:13.564002    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" event={"ID":"1e2b14e7-6838-4c36-95c6-38526a72758d","Type":"ContainerStarted","Data":"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"}
Nov 01 03:24:13.564221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:13.564051    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" event={"ID":"1e2b14e7-6838-4c36-95c6-38526a72758d","Type":"ContainerStarted","Data":"33fdfc2ad0697150a7d5a2e5f596c85dc4c3b566ba272e7cfb23650d5f1b2863"}
Nov 01 03:24:13.681838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:13.681740    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podStartSLOduration=1.681726279 podStartE2EDuration="1.681726279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:24:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:24:13.680124039 +0000 UTC m=+1020.096023569" watchObservedRunningTime="2026-11-01 03:24:13.681726279 +0000 UTC m=+1020.097625870"
Nov 01 03:24:14.016226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:14.016187    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:21.734028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.733991    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:24:21.737897 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.737880    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.800624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.800592    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.800624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.800627    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.800851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.800672    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkmhm\" (UniqueName: \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.800891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.800859    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.800931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.800913    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.801083 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.801057    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.843234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.843204    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"distinct-w1-b12199-kserve-self-signed-certs\""
Nov 01 03:24:21.845420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.845400    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:24:21.903068 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903023    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903200    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903226    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903275    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkmhm\" (UniqueName: \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903332    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903376    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903708    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.903988 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.903949    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.904113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.904042    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.906897 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.906868    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.907543 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.907515    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:21.964974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:21.964946    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkmhm\" (UniqueName: \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\") pod \"distinct-w1-b12199-kserve-644bc7d5db-xhq57\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:22.050452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:22.050411    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:22.475489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:22.475466    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:24:22.477102 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:24:22.477078    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce314bdc_f27e_4a84_b968_7c77c2a51c03.slice/crio-c023a8c01df0fd2203fad84008deb710df477f0b6121c42521e4735a8516c596 WatchSource:0}: Error finding container c023a8c01df0fd2203fad84008deb710df477f0b6121c42521e4735a8516c596: Status 404 returned error can't find the container with id c023a8c01df0fd2203fad84008deb710df477f0b6121c42521e4735a8516c596
Nov 01 03:24:22.620732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:22.620690    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" event={"ID":"ce314bdc-f27e-4a84-b968-7c77c2a51c03","Type":"ContainerStarted","Data":"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"}
Nov 01 03:24:22.620948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:22.620746    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" event={"ID":"ce314bdc-f27e-4a84-b968-7c77c2a51c03","Type":"ContainerStarted","Data":"c023a8c01df0fd2203fad84008deb710df477f0b6121c42521e4735a8516c596"}
Nov 01 03:24:22.738639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:22.738523    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podStartSLOduration=1.73850917 podStartE2EDuration="1.73850917s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:24:22.737326369 +0000 UTC m=+1029.153225888" watchObservedRunningTime="2026-11-01 03:24:22.73850917 +0000 UTC m=+1029.154408734"
Nov 01 03:24:23.015801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:23.015767    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:23.028304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:23.028274    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:23.050545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:23.050520    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:23.636635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:23.636607    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:24:24.647106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.647073    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:24:24.650783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.650760    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.704386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.704355    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-19faa5-openshift-default-dockercfg-tk569\""
Nov 01 03:24:24.744188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744150    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744368 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744308    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744368 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744341    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744540    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744724    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744817 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744758    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744817 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744807    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744834    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.744931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.744849    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5pdf\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.756853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.756818    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:24:24.846333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846294    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846360    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846467    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846500    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846528    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846553    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846580    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5pdf\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846658    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846763    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.846930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846858    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.847070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.846962    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.847176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.847155    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.847254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.847182    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.847254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.847219    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.848753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.848732    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.849056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.849035    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.908069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.907992    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5pdf\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.908205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.908143    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:24.964358 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:24.964320    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:25.405555 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.405528    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:24:25.407531 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:24:25.407492    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55edab28_38c0_4099_859e_1d59ed703620.slice/crio-d9b414d278518fe2542021a4f6326c8782530dd822db91a989d41a01ec1c52f3 WatchSource:0}: Error finding container d9b414d278518fe2542021a4f6326c8782530dd822db91a989d41a01ec1c52f3: Status 404 returned error can't find the container with id d9b414d278518fe2542021a4f6326c8782530dd822db91a989d41a01ec1c52f3
Nov 01 03:24:25.409960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.409930    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:25.410038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.410028    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:25.410075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.410056    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:25.638822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.638785    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" event={"ID":"55edab28-38c0-4099-859e-1d59ed703620","Type":"ContainerStarted","Data":"312952801d4d34707b8232d4cdea43eceee92d2182ad810bcd07a858fe8bafe2"}
Nov 01 03:24:25.639047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.638845    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" event={"ID":"55edab28-38c0-4099-859e-1d59ed703620","Type":"ContainerStarted","Data":"d9b414d278518fe2542021a4f6326c8782530dd822db91a989d41a01ec1c52f3"}
Nov 01 03:24:25.761843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.761792    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podStartSLOduration=1.761777175 podStartE2EDuration="1.761777175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:24:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:24:25.760200323 +0000 UTC m=+1032.176099851" watchObservedRunningTime="2026-11-01 03:24:25.761777175 +0000 UTC m=+1032.177676703"
Nov 01 03:24:25.964819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:25.964785    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:26.967288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:26.967249    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:26.967836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:26.967308    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:27.609789 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:24:27.609752    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:24:27.964925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:27.964829    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:27.964925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:27.964886    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:28.964704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:28.964669    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:28.965232 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:28.964730    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:29.965511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:29.965477    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:29.965905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:29.965532    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:30.965688 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:30.965651    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:30.966112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:30.965708    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:31.965385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:31.965348    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Nov 01 03:24:31.965610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:31.965410    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Nov 01 03:24:32.050940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.050894    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:32.071269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.071236    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:32.697543 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.697515    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:24:32.970334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.970222    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:32.970678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.970656    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:32.971278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:32.971260    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:24:54.642343 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.642312    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:24:54.646299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.646278    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.691937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.691906    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.691948    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.691990    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692207 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692115    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692207 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692171    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692222    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692266    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmq9j\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692354 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692328    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.692396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.692362    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793189    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793231    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793272    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793292    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793316    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793366    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793403    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793469    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793492    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmq9j\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793732    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.793927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.793888    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.794168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.794145    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.794263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.794152    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.794263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.794198    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.795576 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.795550    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.795791 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.795772    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.851907 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.851875    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:24:54.855916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.855888    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.857180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.857153    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmq9j\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\") pod \"e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") " pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:54.963187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:54.963101    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:55.395960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.395925    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:24:55.433136 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:24:55.433106    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a5a51fb_e4f0_42bb_b501_00769e91ed80.slice/crio-d53f9b569ff0578b9708f781b29c387510143f9c12c7a231df58522e923241ae WatchSource:0}: Error finding container d53f9b569ff0578b9708f781b29c387510143f9c12c7a231df58522e923241ae: Status 404 returned error can't find the container with id d53f9b569ff0578b9708f781b29c387510143f9c12c7a231df58522e923241ae
Nov 01 03:24:55.435353 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.435318    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:55.435438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.435387    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:55.435438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.435415    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:55.814515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.814483    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" event={"ID":"6a5a51fb-e4f0-42bb-b501-00769e91ed80","Type":"ContainerStarted","Data":"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"}
Nov 01 03:24:55.815010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.814543    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" event={"ID":"6a5a51fb-e4f0-42bb-b501-00769e91ed80","Type":"ContainerStarted","Data":"d53f9b569ff0578b9708f781b29c387510143f9c12c7a231df58522e923241ae"}
Nov 01 03:24:55.949723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.949650    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podStartSLOduration=1.949627642 podStartE2EDuration="1.949627642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:24:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:24:55.938224259 +0000 UTC m=+1062.354123791" watchObservedRunningTime="2026-11-01 03:24:55.949627642 +0000 UTC m=+1062.365527170"
Nov 01 03:24:55.964059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:55.964019    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:24:56.550565 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.550529    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:24:56.554558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.554537    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.607235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.607207    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-e8c789-openshift-default-dockercfg-vpp4p\""
Nov 01 03:24:56.615926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.615886    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.615926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.615926    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616051    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616114    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616228    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616263    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616337    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616428 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616371    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.616428 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.616409    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjg7p\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.660318 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.660283    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:24:56.717257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717220    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717265    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717301    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717342    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717449    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjg7p\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717529    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717567    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717637    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717654    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717686    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.717768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717745    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.718060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717883    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.718060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.717942    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.718361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.718337    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.719796 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.719775    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.719893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.719847    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.798607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.798575    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.812025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.811906    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjg7p\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.870763 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.870725    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:56.969000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.968601    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:24:56.969000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:56.968674    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:24:57.454004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.450403    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:24:57.466534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.466493    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:57.466627 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.466588    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:57.466682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.466635    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:24:57.616088 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:24:57.616050    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:24:57.865002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.864556    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" event={"ID":"ca4e8f78-70a6-449d-9dff-86629bb521b4","Type":"ContainerStarted","Data":"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"}
Nov 01 03:24:57.865002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.864618    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" event={"ID":"ca4e8f78-70a6-449d-9dff-86629bb521b4","Type":"ContainerStarted","Data":"3a3da9194f64ee1426f9abae23dfac5695735b9863dab3609f0ece11fadbbede"}
Nov 01 03:24:57.885027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.875651    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:24:57.967003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.966362    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:24:57.967003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:57.966421    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:24:58.010414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:58.010360    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podStartSLOduration=2.010343998 podStartE2EDuration="2.010343998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:24:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:24:58.00914064 +0000 UTC m=+1064.425040170" watchObservedRunningTime="2026-11-01 03:24:58.010343998 +0000 UTC m=+1064.426243579"
Nov 01 03:24:58.892039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:58.891468    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:24:58.892039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:58.891540    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:24:58.983002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:58.966368    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:24:58.983002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:58.966429    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:24:59.873506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:59.873468    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:24:59.874498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:59.873783    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:24:59.968000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:59.967352    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:24:59.968000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:24:59.967415    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:25:00.871825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:00.871770    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:25:00.872046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:00.871839    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:25:00.971458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:00.971421    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:25:00.972107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:00.972076    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:25:01.872032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:01.871992    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:25:01.872216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:01.872066    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:25:01.964379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:01.964337    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Nov 01 03:25:01.964553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:01.964409    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Nov 01 03:25:02.871251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:02.871213    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:25:02.871736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:02.871286    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:25:02.967863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:02.967834    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:25:02.968306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:02.968279    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:25:02.969000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:02.968955    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:25:03.872159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:03.872120    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Nov 01 03:25:03.872606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:03.872193    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Nov 01 03:25:04.875913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:04.875885    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:25:04.876419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:04.876208    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:25:04.877025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:04.877008    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:25:10.461926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.461888    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:25:10.465863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.465837    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.566229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.564570    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:25:10.583625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583581    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.583815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583695    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.583815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583724    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48h5x\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.583815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583789    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.584042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583871    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.584042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.583897    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.584042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.584039    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.584224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.584062    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.584224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.584190    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.684951    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685034    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685059    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685108    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685134    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685203    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685229    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685289    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685310    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48h5x\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685611    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685702    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685733    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.685940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.685895    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.686090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.686064    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.687856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.687833    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.688290 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.688267    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.748271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.748189    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.748793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.748768    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-48h5x\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.772534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.772503    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:25:10.777384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.777363    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.780554 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.780533    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:10.845011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.842892    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:25:10.887780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.887743    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.887932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.887857    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.887932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.887893    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.887952    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.887989    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.888026    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.888071    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kp56\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888143 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.888115    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.888174 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.888154    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.989807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.989773    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.989807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.989809    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.989844    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.989888    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kp56\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990169 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990033    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990169 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990102    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990171    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990268    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990304    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990331    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990348    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990397    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990453    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.990945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.990922    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.992522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.992500    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:10.992737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:10.992720    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:11.051622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.051590    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:11.051801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.051708    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kp56\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:11.092633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.092598    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:11.220260 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.220233    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:25:11.222088 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:25:11.222054    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42581529_bc07_45a9_9bf1_1c45989b4bf6.slice/crio-da47911bd5fca0473e77897f03ffbe1a633e28a5783495458620847dd60519c4 WatchSource:0}: Error finding container da47911bd5fca0473e77897f03ffbe1a633e28a5783495458620847dd60519c4: Status 404 returned error can't find the container with id da47911bd5fca0473e77897f03ffbe1a633e28a5783495458620847dd60519c4
Nov 01 03:25:11.225026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.224954    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:11.225128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.225071    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:11.225128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.225110    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:11.477666 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:25:11.477635    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ade366_fa02_4a60_b751_9e652e53f260.slice/crio-b9a63bd9f597b26dcba6f4392e9fccfbe2bbce18d6652e8963221ef19ff1f5de WatchSource:0}: Error finding container b9a63bd9f597b26dcba6f4392e9fccfbe2bbce18d6652e8963221ef19ff1f5de: Status 404 returned error can't find the container with id b9a63bd9f597b26dcba6f4392e9fccfbe2bbce18d6652e8963221ef19ff1f5de
Nov 01 03:25:11.478218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.478192    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:25:11.480367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.480327    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:11.480452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.480411    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:11.480489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:11.480465    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:12.057004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.056181    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" event={"ID":"42581529-bc07-45a9-9bf1-1c45989b4bf6","Type":"ContainerStarted","Data":"874048c70c6cc89c7d0bb141cea3aa4d4347bc15edb68b67ab7d811d20b9460d"}
Nov 01 03:25:12.057004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.056266    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" event={"ID":"42581529-bc07-45a9-9bf1-1c45989b4bf6","Type":"ContainerStarted","Data":"da47911bd5fca0473e77897f03ffbe1a633e28a5783495458620847dd60519c4"}
Nov 01 03:25:12.078002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.064739    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" event={"ID":"d6ade366-fa02-4a60-b751-9e652e53f260","Type":"ContainerStarted","Data":"f1ab6ca8d782099ab1e4f5a4dc77c0d9295265ff2c832c4b6f22b29a4923933c"}
Nov 01 03:25:12.078002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.064809    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" event={"ID":"d6ade366-fa02-4a60-b751-9e652e53f260","Type":"ContainerStarted","Data":"b9a63bd9f597b26dcba6f4392e9fccfbe2bbce18d6652e8963221ef19ff1f5de"}
Nov 01 03:25:12.092749 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.092717    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:12.094855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.094829    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:12.095055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.095030    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:12.180914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.180849    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podStartSLOduration=2.180830308 podStartE2EDuration="2.180830308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:25:12.178872243 +0000 UTC m=+1078.594771775" watchObservedRunningTime="2026-11-01 03:25:12.180830308 +0000 UTC m=+1078.596729842"
Nov 01 03:25:12.315372 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.315274    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podStartSLOduration=2.315259366 podStartE2EDuration="2.315259366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:25:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:25:12.313841392 +0000 UTC m=+1078.729740919" watchObservedRunningTime="2026-11-01 03:25:12.315259366 +0000 UTC m=+1078.731158894"
Nov 01 03:25:12.781158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.781109    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:12.782682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.782656    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:12.782804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:12.782708    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:13.093398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:13.093320    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:13.093398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:13.093382    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:13.781228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:13.781197    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:13.781702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:13.781679    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:14.092966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:14.092875    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:14.092966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:14.092923    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:14.783596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:14.781396    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:14.783596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:14.781458    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:15.093836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:15.093745    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:15.093836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:15.093799    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:15.781636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:15.781595    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:15.781824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:15.781662    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:16.095113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:16.095029    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:16.095113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:16.095089    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:16.781029 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:16.780958    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:16.781211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:16.781047    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:17.093972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:17.093881    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:17.093972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:17.093942    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:17.781438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:17.781391    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Nov 01 03:25:17.781842 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:17.781472    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Nov 01 03:25:18.093577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:18.093480    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Nov 01 03:25:18.093577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:18.093542    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Nov 01 03:25:18.785734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:18.785699    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:18.786244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:18.786078    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:18.786838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:18.786818    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:25:19.099702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:19.099606    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:19.100192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:19.100166    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:19.101214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:19.101187    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:25:27.617198 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:25:27.617114    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:25:55.763967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.763930    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:25:55.769371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.769347    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812326    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812378    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812434    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812465    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812489    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812562    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812756    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812813    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.812889 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.812878    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.879096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.879061    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:25:55.914007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.913952    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914057    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914095    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914162    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914212    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914243    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914297    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914335    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914372    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914461    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914506    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914761 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914733    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.914900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.914878    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.915046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.915026    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.916532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.916515    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.916756 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.916733    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.985569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.985535    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:55.986032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:55.986008    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\") pod \"e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") " pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:56.083054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.082932    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:56.520635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.520601    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:25:56.522050 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:25:56.522026    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod119d01d7_b601_495c_8945_90ddc2110f5d.slice/crio-eedfbf4028d22cb7c75e9725a74bb2911010a0167b5ec42247db369cc73f4c03 WatchSource:0}: Error finding container eedfbf4028d22cb7c75e9725a74bb2911010a0167b5ec42247db369cc73f4c03: Status 404 returned error can't find the container with id eedfbf4028d22cb7c75e9725a74bb2911010a0167b5ec42247db369cc73f4c03
Nov 01 03:25:56.524359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.524329    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:56.524456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.524391    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:56.524456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.524421    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:56.828526 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.828443    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:25:56.832508 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.832483    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.914823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.914788    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:25:56.926947 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.926909    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927011    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927075    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927216    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927263    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927292    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927331    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927354    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:56.927495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:56.927448    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dlhb\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028599    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028660    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028691    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028724    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028747    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.028861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028830    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dlhb\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028869    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.028924    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029031    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029414    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029511    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029627    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029690    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.029782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.029748    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.031659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.031637    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.031867 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.031851    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.091285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.091210    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.091438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.091322    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dlhb\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.153003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.148604    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:57.365474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.365377    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" event={"ID":"119d01d7-b601-495c-8945-90ddc2110f5d","Type":"ContainerStarted","Data":"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"}
Nov 01 03:25:57.365474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.365469    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" event={"ID":"119d01d7-b601-495c-8945-90ddc2110f5d","Type":"ContainerStarted","Data":"eedfbf4028d22cb7c75e9725a74bb2911010a0167b5ec42247db369cc73f4c03"}
Nov 01 03:25:57.489642 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.489587    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podStartSLOduration=2.489571694 podStartE2EDuration="2.489571694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:25:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:25:57.487694553 +0000 UTC m=+1123.903594086" watchObservedRunningTime="2026-11-01 03:25:57.489571694 +0000 UTC m=+1123.905471223"
Nov 01 03:25:57.598167 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.598136    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:25:57.598941 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:25:57.598914    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod965808b9_89ff_4911_837d_bd3f9b2b4527.slice/crio-a4ed5c87cbb0c8241fef76ed83dd04d97c90fad06e04349a0874e1020ce1ad8f WatchSource:0}: Error finding container a4ed5c87cbb0c8241fef76ed83dd04d97c90fad06e04349a0874e1020ce1ad8f: Status 404 returned error can't find the container with id a4ed5c87cbb0c8241fef76ed83dd04d97c90fad06e04349a0874e1020ce1ad8f
Nov 01 03:25:57.601596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.601563    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:57.601723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.601626    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:57.601723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:57.601655    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:25:57.612642 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:25:57.612612    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:25:58.083973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.083938    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:25:58.085766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.085738    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:25:58.085910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.085783    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:25:58.377894 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.377750    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" event={"ID":"965808b9-89ff-4911-837d-bd3f9b2b4527","Type":"ContainerStarted","Data":"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"}
Nov 01 03:25:58.377894 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.377819    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" event={"ID":"965808b9-89ff-4911-837d-bd3f9b2b4527","Type":"ContainerStarted","Data":"a4ed5c87cbb0c8241fef76ed83dd04d97c90fad06e04349a0874e1020ce1ad8f"}
Nov 01 03:25:58.502520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:58.502463    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podStartSLOduration=2.502445405 podStartE2EDuration="2.502445405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:25:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:25:58.499162291 +0000 UTC m=+1124.915061822" watchObservedRunningTime="2026-11-01 03:25:58.502445405 +0000 UTC m=+1124.918344936"
Nov 01 03:25:59.084077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:59.084041    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:25:59.084489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:59.084099    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:25:59.154003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:59.153394    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:25:59.154003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:59.153550    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:25:59.154003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:25:59.153599    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:00.083754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:00.083709    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:26:00.084023 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:00.083782    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:26:00.150010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:00.149308    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:26:00.150010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:00.149369    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:01.084217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:01.084181    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:26:01.084406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:01.084232    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:26:01.156004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:01.155804    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:26:01.156004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:01.155861    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:02.083869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:02.083827    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:26:02.084082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:02.083906    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:26:02.151009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:02.150404    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:26:02.151009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:02.150483    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:03.083931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:03.083888    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Nov 01 03:26:03.084384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:03.083946    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Nov 01 03:26:03.150524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:03.150405    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:26:03.150524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:03.150467    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:04.149678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:04.149639    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Nov 01 03:26:04.150199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:04.149728    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Nov 01 03:26:04.158184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:04.158153    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:26:04.158521 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:04.158500    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:26:04.159440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:04.159420    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:26:05.323596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:05.323547    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:26:05.324141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:05.324049    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:26:05.325503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:05.325476    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:26:27.610390 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:26:27.610348    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:26:33.737783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.737749    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:26:33.741397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.741378    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.846489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.846443    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-b12199\"/\"distinct-2-w1-b12199-kserve-self-signed-certs\""
Nov 01 03:26:33.849409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.849386    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:26:33.858539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858515    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.858619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858560    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.858686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858665    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.858730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858697    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.858873 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858853    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp5n6\" (UniqueName: \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.859051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.858917    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960396    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp5n6\" (UniqueName: \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960455    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960498    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960525    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960573    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960679    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.960950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960910    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.961027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.960953    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.961243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.961213    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.962967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.962937    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:33.963229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:33.963210    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:34.027069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.027030    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp5n6\" (UniqueName: \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\") pod \"distinct-2-w1-b12199-kserve-7579877bb9-v6xcb\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") " pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:34.053729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.053690    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:34.445126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.445096    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:26:34.446631 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:26:34.446598    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f5f6fc7_8cf1_4c2e_a6fc_d3dadd2c5348.slice/crio-5025057db627fc75dd7945dbe5fecc146b7b5c23b43e79c55d37c764b94fcb17 WatchSource:0}: Error finding container 5025057db627fc75dd7945dbe5fecc146b7b5c23b43e79c55d37c764b94fcb17: Status 404 returned error can't find the container with id 5025057db627fc75dd7945dbe5fecc146b7b5c23b43e79c55d37c764b94fcb17
Nov 01 03:26:34.698271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.698173    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" event={"ID":"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348","Type":"ContainerStarted","Data":"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"}
Nov 01 03:26:34.698271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.698210    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" event={"ID":"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348","Type":"ContainerStarted","Data":"5025057db627fc75dd7945dbe5fecc146b7b5c23b43e79c55d37c764b94fcb17"}
Nov 01 03:26:34.817771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:34.817716    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podStartSLOduration=1.817700264 podStartE2EDuration="1.817700264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:26:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:26:34.816367638 +0000 UTC m=+1161.232267167" watchObservedRunningTime="2026-11-01 03:26:34.817700264 +0000 UTC m=+1161.233599849"
Nov 01 03:26:35.054052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:35.054015    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:44.057005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:44.056105    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:44.235651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:44.235617    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:44.785320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:44.785291    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:26:57.615180 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:26:57.614080    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:26:59.948505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:59.948473    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:26:59.952294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:26:59.952272    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.014482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.014448    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:27:00.029629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029592    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.029629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029639    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zccdf\" (UniqueName: \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.029901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029799    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.029901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029859    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.030065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029914    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.030065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.029966    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.060927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.060890    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d7f535\"/\"distinct-2-w3-d7f535-kserve-self-signed-certs\""
Nov 01 03:27:00.133300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.133259    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.133599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.133578    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.133935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.133912    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.134560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.134539    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.134911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.134892    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.136819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.136764    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.136961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.136765    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zccdf\" (UniqueName: \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.137813 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.137792    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.138398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.138368    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.150739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.150701    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.156230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.156192    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.210696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.210614    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zccdf\" (UniqueName: \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\") pod \"distinct-2-w3-d7f535-kserve-c48846f65-dtg44\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") " pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.265649 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.265607    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:00.677360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.677325    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:27:00.680902 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:27:00.680847    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a22c5f8_69c6_4e75_ada2_f3fa2b321bdd.slice/crio-aabe23cc22201dee9de6210d28e4eb19f2c19bb43d5bff38a041fbb5fd2a2964 WatchSource:0}: Error finding container aabe23cc22201dee9de6210d28e4eb19f2c19bb43d5bff38a041fbb5fd2a2964: Status 404 returned error can't find the container with id aabe23cc22201dee9de6210d28e4eb19f2c19bb43d5bff38a041fbb5fd2a2964
Nov 01 03:27:00.930060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.929609    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" event={"ID":"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd","Type":"ContainerStarted","Data":"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"}
Nov 01 03:27:00.930060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:00.929665    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" event={"ID":"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd","Type":"ContainerStarted","Data":"aabe23cc22201dee9de6210d28e4eb19f2c19bb43d5bff38a041fbb5fd2a2964"}
Nov 01 03:27:01.057544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:01.057481    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podStartSLOduration=2.057464626 podStartE2EDuration="2.057464626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:26:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:27:01.057231138 +0000 UTC m=+1187.473130671" watchObservedRunningTime="2026-11-01 03:27:01.057464626 +0000 UTC m=+1187.473364155"
Nov 01 03:27:01.265757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:01.265718    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:03.520444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.520399    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:27:03.645158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.645109    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 01 03:27:03.683164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.683121    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:03.706263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.706208    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:27:03.725422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.707105    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:03.781218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.781113    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:03.840104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:03.840055    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:27:04.027005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:04.023063    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:10.266729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:10.266680    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:10.279719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:10.279685    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:11.032800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:11.032767    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:27:12.635634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.635599    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:27:12.639827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.639803    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686010    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686056    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686155    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686202    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686348 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686333    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686350    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs5mv\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686375    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686489    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.686609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.686593    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.745874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.745836    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:27:12.787941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.787903    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.787949    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs5mv\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788050    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788100    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788247    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788315    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788356    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788427    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788479    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.788919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.788872    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.789454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.789429    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.789693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.789667    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.789893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.789834    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.790283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.790259    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.791319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.791296    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.792199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.792167    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.853353 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.853314    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.853515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.853419    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs5mv\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:12.953545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:12.953459    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:13.389944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.389913    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:27:13.390995 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:27:13.390951    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5886cdb_de25_4b50_a6e0_adb1e826c943.slice/crio-177635b6508e64d322a6f3ab982b5c5faad29c1558e2e427a4d41dabdf6c9069 WatchSource:0}: Error finding container 177635b6508e64d322a6f3ab982b5c5faad29c1558e2e427a4d41dabdf6c9069: Status 404 returned error can't find the container with id 177635b6508e64d322a6f3ab982b5c5faad29c1558e2e427a4d41dabdf6c9069
Nov 01 03:27:13.393527 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.393495    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:13.393600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.393560    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:13.393600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.393588    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:13.820393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.820362    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:27:13.822307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:13.822278    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:27:14.049472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.049436    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" event={"ID":"f5886cdb-de25-4b50-a6e0-adb1e826c943","Type":"ContainerStarted","Data":"a9144291f5b13cacc7631e69b36c8588151d53623e315e2fef4fbdad375e889d"}
Nov 01 03:27:14.049659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.049494    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" event={"ID":"f5886cdb-de25-4b50-a6e0-adb1e826c943","Type":"ContainerStarted","Data":"177635b6508e64d322a6f3ab982b5c5faad29c1558e2e427a4d41dabdf6c9069"}
Nov 01 03:27:14.225415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.225307    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podStartSLOduration=2.225291617 podStartE2EDuration="2.225291617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:27:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:27:14.224723906 +0000 UTC m=+1200.640623436" watchObservedRunningTime="2026-11-01 03:27:14.225291617 +0000 UTC m=+1200.641191578"
Nov 01 03:27:14.953803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.953763    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:14.955283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.955259    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:14.955345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:14.955328    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:15.954608 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:15.954570    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:15.955071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:15.954650    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:16.954261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:16.954217    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:16.954453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:16.954315    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:17.954656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:17.954617    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:17.955124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:17.954715    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:18.953994 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:18.953944    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:18.954183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:18.954072    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:19.954887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:19.954848    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Nov 01 03:27:19.955468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:19.954914    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Nov 01 03:27:21.977003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:21.967070    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:27:21.977003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:21.967137    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:22.101015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.099325    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:22.101015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.100814    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:22.130010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.120158    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:27:22.599424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.598545    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:27:22.627898 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.627077    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 01 03:27:22.644464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.644113    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:22.665675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.665635    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:22.696951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.696096    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:27:22.780216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.780173    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:27:22.838114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:22.838061    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:23.028993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.028426    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:23.055670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.055552    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:23.612003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.605115    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:27:23.612003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.605568    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:27:23.632819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.632069    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 01 03:27:23.632819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.632230    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:27:23.751924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.751890    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:27:23.772699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.772670    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:27:23.814686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.814375    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:27:23.866868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.866782    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:27:23.867607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:23.867589    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:27:27.612595 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:27:27.612557    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:27:41.962865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:41.962826    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:27:41.966836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:41.966816    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:41.984241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:41.984211    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:27:41.988226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:41.988205    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.089524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089480    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.089733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089596    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.089733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089649    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.089733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089686    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.089945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089830    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.089945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089901    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.089945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.089932    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.090148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090097    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090145    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090168    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.090268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090184    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.090268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090251    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090278    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7g4c\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090322    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8mk6\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090351    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090380    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090446    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.090811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.090493    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.103838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.103805    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:27:42.173571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.173528    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:27:42.192244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192202    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192281    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192328    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192385    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192435    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192463    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192527    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192572    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192601    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192623    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192673    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192684    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192713    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7g4c\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192761    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8mk6\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192791    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192824    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192896    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.192927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.192921    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.193375 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.193014    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.194240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.193456    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.194240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.193809    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.194240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.193865    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.194240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.194169    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.194543 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.194423    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.194596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.194568    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.195214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.194837    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.195214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.194901    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.195896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.195574    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.195896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.195858    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.197393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.197368    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.197822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.197797    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.197940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.197875    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.256453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.256420    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.256635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.256470    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.256635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.256480    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8mk6\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.256635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.256562    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7g4c\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\") pod \"e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") " pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.280853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.280821    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:42.301477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.301237    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:42.704593 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.704510    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:27:42.709798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.709775    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.733451 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.733419    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:27:42.734281 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:27:42.734255    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c7a2a0f_1a45_448e_b0b9_3c8b2d26e4f0.slice/crio-53b7255661f2921887bf7b1b6752564b1429b5a8ad51b6a3967e388e84d1d8a7 WatchSource:0}: Error finding container 53b7255661f2921887bf7b1b6752564b1429b5a8ad51b6a3967e388e84d1d8a7: Status 404 returned error can't find the container with id 53b7255661f2921887bf7b1b6752564b1429b5a8ad51b6a3967e388e84d1d8a7
Nov 01 03:27:42.737050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.737018    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.737142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.737082    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.737142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.737112    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.743922 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.743897    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:27:42.764624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.764108    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.764624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.764190    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.764624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.764239    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:42.773637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.773609    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:27:42.802347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802310    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802445    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802606    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802749 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802735    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802767    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802801    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802833    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802873    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8rk\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.802915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.802905    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.904607    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8rk\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.904662    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.904704    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.904777    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.904894    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905035    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905064    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905097    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905127    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905425    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905462    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.905675    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.906128    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.906737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.906379    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.908871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.908623    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.908871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.908822    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.968102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.968030    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:42.968441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:42.968122    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8rk\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\") pod \"e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") " pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:43.023733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.023691    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:43.314694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.314662    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" event={"ID":"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0","Type":"ContainerStarted","Data":"8a4713369a7468c0511c5b2f04a2d1fbabd826241594ade31e5277018ee31d11"}
Nov 01 03:27:43.314877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.314754    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" event={"ID":"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0","Type":"ContainerStarted","Data":"53b7255661f2921887bf7b1b6752564b1429b5a8ad51b6a3967e388e84d1d8a7"}
Nov 01 03:27:43.316413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.316387    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" event={"ID":"2e16e809-93d0-4fa3-86c0-770c32921326","Type":"ContainerStarted","Data":"21e562952fb8d7bb0dc18716b0289c4053e9186e999ff7a64fcd040da393cd8e"}
Nov 01 03:27:43.316524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.316429    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" event={"ID":"2e16e809-93d0-4fa3-86c0-770c32921326","Type":"ContainerStarted","Data":"d35277d0cf1e5c1186150c3ec9d61f1ac572aa1690947e99e61e3b7091411d7f"}
Nov 01 03:27:43.449040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.448769    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:27:43.451687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.451640    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podStartSLOduration=2.451627529 podStartE2EDuration="2.451627529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:27:43.449106028 +0000 UTC m=+1229.865005557" watchObservedRunningTime="2026-11-01 03:27:43.451627529 +0000 UTC m=+1229.867527057"
Nov 01 03:27:43.452296 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:27:43.452262    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2f2e9be_83a5_416f_96d5_4626208aad4e.slice/crio-76ace56c387de2ab68ef86c92220f4ba399ac29942e0b10b0374e6569c9e010a WatchSource:0}: Error finding container 76ace56c387de2ab68ef86c92220f4ba399ac29942e0b10b0374e6569c9e010a: Status 404 returned error can't find the container with id 76ace56c387de2ab68ef86c92220f4ba399ac29942e0b10b0374e6569c9e010a
Nov 01 03:27:43.455082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.455051    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:43.455170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.455119    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:43.455170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.455147    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:27:43.577040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:43.576896    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podStartSLOduration=2.576878361 podStartE2EDuration="2.576878361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:27:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:27:43.57417282 +0000 UTC m=+1229.990072349" watchObservedRunningTime="2026-11-01 03:27:43.576878361 +0000 UTC m=+1229.992777890"
Nov 01 03:27:44.281716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.281676    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:44.283099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.283064    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:44.283171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.283129    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:44.301618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.301591    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:44.303019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.302990    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:44.303124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.303031    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:44.376102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.376059    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" event={"ID":"e2f2e9be-83a5-416f-96d5-4626208aad4e","Type":"ContainerStarted","Data":"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"}
Nov 01 03:27:44.376278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.376118    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" event={"ID":"e2f2e9be-83a5-416f-96d5-4626208aad4e","Type":"ContainerStarted","Data":"76ace56c387de2ab68ef86c92220f4ba399ac29942e0b10b0374e6569c9e010a"}
Nov 01 03:27:44.519213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:44.516719    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podStartSLOduration=2.516698636 podStartE2EDuration="2.516698636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:27:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:27:44.515313568 +0000 UTC m=+1230.931213097" watchObservedRunningTime="2026-11-01 03:27:44.516698636 +0000 UTC m=+1230.932598167"
Nov 01 03:27:45.033010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.032797    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:45.080005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.065180    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:45.080005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.065260    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:45.282004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.281409    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:45.282004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.281483    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:45.302695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.302594    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:45.302695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:45.302659    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:46.029005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.026133    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:46.029005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.026195    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:46.304059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.292394    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:46.304059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.292473    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:46.304059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.302376    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:46.304059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:46.302446    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:47.036013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.035096    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:47.036013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.035163    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:47.297457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.297373    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:47.298139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.298106    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:47.304420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.304303    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:47.304420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:47.304361    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:48.077360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.077314    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:48.081031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.077621    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:48.310110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.305830    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:48.310110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.305891    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:48.310110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.307446    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:48.310110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:48.307502    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:49.043006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.040011    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:49.043006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.040083    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:49.286048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.282472    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:49.286048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.282537    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:49.302396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.302307    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:49.302396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:49.302355    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:50.036004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.027090    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:50.036004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.027167    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:50.282086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.282045    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Nov 01 03:27:50.282303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.282103    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Nov 01 03:27:50.303820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.303631    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Nov 01 03:27:50.303820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:50.303694    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Nov 01 03:27:51.024234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.024193    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Nov 01 03:27:51.024518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.024261    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Nov 01 03:27:51.295527 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.295374    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:51.296360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.296335    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:51.297361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.297339    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:27:51.308467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.308437    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:51.308780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.308759    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:51.309764 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:51.309739    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:27:52.029474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:52.029442    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:52.029867 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:52.029843    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:52.030608 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:52.030589    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:27:54.858743 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:27:54.858694    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:27:57.611891 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:27:57.611855    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:28:11.125610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.125572    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"]
Nov 01 03:28:11.129735 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.129712    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.174225 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.174192    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbfn5\" (UniqueName: \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.174396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.174249    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.174601 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.174572    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.233862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.233831    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:28:11.233862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.233829    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:28:11.237524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.237498    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"]
Nov 01 03:28:11.276261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.276223    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbfn5\" (UniqueName: \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.276261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.276269    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.276466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.276427    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.277022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.276954    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.278718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.278695    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.338389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.338354    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbfn5\" (UniqueName: \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\") pod \"maas-api-revoke-sub-6e66f2ea-dlnqt\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.443284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.443201    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:11.485328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.485294    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:28:11.490034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.489998    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.572192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.572161    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:28:11.580830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.580791    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.580830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.580837    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.580951    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581053    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581135    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5vhv\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581252    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581301    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581418    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.581470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.581449    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682431    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5vhv\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682500    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682530    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682605    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682638    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682802    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682834    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.682914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682884    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.682918    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.683020    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.683278    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.683436    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683617 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.683581    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.683888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.683867    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.685188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.685166    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.685299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.685282    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.743878 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.743803    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.744050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.744010    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5vhv\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\") pod \"e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") " pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.807583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.807545    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:11.830743 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.830716    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"]
Nov 01 03:28:11.833535 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:11.833502    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37e1014_0507_4902_a61e_d3f86f0796eb.slice/crio-5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162 WatchSource:0}: Error finding container 5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162: Status 404 returned error can't find the container with id 5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162
Nov 01 03:28:11.835867 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.835846    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:28:11.896391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:11.892526    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt" event={"ID":"e37e1014-0507-4902-a61e-d3f86f0796eb","Type":"ContainerStarted","Data":"5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162"}
Nov 01 03:28:12.200966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.200928    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:28:12.202575 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:12.202538    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f03dc66_e6b6_44e0_813b_364d5061ff33.slice/crio-29d8a17f835bbc1712fb0ddeb313e6bf5f8ffc9ed761f3d9c748423fb6d24fd4 WatchSource:0}: Error finding container 29d8a17f835bbc1712fb0ddeb313e6bf5f8ffc9ed761f3d9c748423fb6d24fd4: Status 404 returned error can't find the container with id 29d8a17f835bbc1712fb0ddeb313e6bf5f8ffc9ed761f3d9c748423fb6d24fd4
Nov 01 03:28:12.205446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.205410    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:12.205549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.205482    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:12.205549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.205512    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:12.903218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.903170    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" event={"ID":"2f03dc66-e6b6-44e0-813b-364d5061ff33","Type":"ContainerStarted","Data":"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"}
Nov 01 03:28:12.903414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:12.903286    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" event={"ID":"2f03dc66-e6b6-44e0-813b-364d5061ff33","Type":"ContainerStarted","Data":"29d8a17f835bbc1712fb0ddeb313e6bf5f8ffc9ed761f3d9c748423fb6d24fd4"}
Nov 01 03:28:13.137052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:13.134705    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podStartSLOduration=2.134682881 podStartE2EDuration="2.134682881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:13.074371632 +0000 UTC m=+1259.490271160" watchObservedRunningTime="2026-11-01 03:28:13.134682881 +0000 UTC m=+1259.550582412"
Nov 01 03:28:13.808376 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:13.808340    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:13.809731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:13.809685    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:13.810200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:13.810171    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:14.808324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:14.808291    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:14.808530 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:14.808344    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:15.808411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:15.808378    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:15.808581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:15.808457    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:16.808505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:16.808475    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:16.808679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:16.808570    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:17.808280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:17.808243    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:17.808459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:17.808298    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:17.960364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:17.960328    4892 generic.go:356] "Generic (PLEG): container finished" podID="e37e1014-0507-4902-a61e-d3f86f0796eb" containerID="f3e53660ce95cd4c043a9754167de87ef2eea35709c3da229c7c80e8bcd264c1" exitCode=0
Nov 01 03:28:17.960728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:17.960363    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt" event={"ID":"e37e1014-0507-4902-a61e-d3f86f0796eb","Type":"ContainerDied","Data":"f3e53660ce95cd4c043a9754167de87ef2eea35709c3da229c7c80e8bcd264c1"}
Nov 01 03:28:18.809015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:18.808936    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Nov 01 03:28:18.810390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:18.810353    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Nov 01 03:28:19.164162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.164134    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:19.226273 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.226228    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\") pod \"e37e1014-0507-4902-a61e-d3f86f0796eb\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") "
Nov 01 03:28:19.226430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.226397    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\") pod \"e37e1014-0507-4902-a61e-d3f86f0796eb\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") "
Nov 01 03:28:19.226507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.226469    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\" (UniqueName: \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\") pod \"e37e1014-0507-4902-a61e-d3f86f0796eb\" (UID: \"e37e1014-0507-4902-a61e-d3f86f0796eb\") "
Nov 01 03:28:19.226725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.226681    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca" pod "e37e1014-0507-4902-a61e-d3f86f0796eb" (UID: "e37e1014-0507-4902-a61e-d3f86f0796eb"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:28:19.227263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.227240    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:19.228467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.228446    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca" pod "e37e1014-0507-4902-a61e-d3f86f0796eb" (UID: "e37e1014-0507-4902-a61e-d3f86f0796eb"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:28:19.229074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.229045    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5" pod "e37e1014-0507-4902-a61e-d3f86f0796eb" (UID: "e37e1014-0507-4902-a61e-d3f86f0796eb"). InnerVolumeSpecName "kube-api-access-dbfn5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:28:19.328772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.328729    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e37e1014-0507-4902-a61e-d3f86f0796eb-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:19.328772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.328776    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbfn5\" (UniqueName: \"kubernetes.io/projected/e37e1014-0507-4902-a61e-d3f86f0796eb-kube-api-access-dbfn5\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:19.812672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.812639    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:19.813095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.813076    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:19.813937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.813914    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:28:19.976291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.976210    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"
Nov 01 03:28:19.976291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.976239    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt" event={"ID":"e37e1014-0507-4902-a61e-d3f86f0796eb","Type":"ContainerDied","Data":"5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162"}
Nov 01 03:28:19.976495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:19.976292    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5802ba0193b3b884167ca65ec8d880f199044b77da952293b366456910592162"
Nov 01 03:28:26.669264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.669163    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:28:26.696442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.696361    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:28:26.756590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.756543    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:26.778731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.777088    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:26.778731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.777563    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:28:26.838956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:26.838078    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.028759 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.028719    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:28:27.117136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.117081    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.182140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.182087    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.476214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.475086    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.679005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.672071    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.679005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.672993    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:28:27.727079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.726922    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.757017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.747315    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.757700    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.785557    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.786970    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.787056    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.785592    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:28:27.795006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.788959    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:28:27.903219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.893441    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:27.903219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:27.893907    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:28:28.053259 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:28:28.053222    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:28:28.108047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.083139    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:28:28.108047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.083663    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:28:28.160628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.150065    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.160628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.150608    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:28.190299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.184090    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 01 03:28:28.190299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.184591    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:28:28.477221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.477115    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.479306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.479276    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:28:28.734060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.733217    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.742004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.738506    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.807003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.787150    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 01 03:28:28.807003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.798180    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.809241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.809202    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:28.976005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:28.968614    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.101370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.101329    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:28:29.166520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.166477    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.188267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.185524    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 01 03:28:29.478470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.478371    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.774743 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.774691    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:28:29.903492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.785951    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.904145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.804444    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.911008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.811131    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.911805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.834690    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.969511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.969463    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:29.970178 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:29.969796    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:30.101313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.101112    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:28:30.101313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.101181    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:30.102095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.101680    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:28:30.167118 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.167067    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 01 03:28:30.241004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.215095    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:30.494494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.494386    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:30.553479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.553440    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:30.586755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.586714    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:30.916640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.916583    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.017619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.942175    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.017619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:30.942217    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:28:31.041025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.018173    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.041025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.018327    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.041025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.025409    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.041025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.025515    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:28:31.125013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.122624    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.226523 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.226398    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.483062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.482218    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.518047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.501164    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.954013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.952254    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:28:31.977450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.977396    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:31.978106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:31.978069    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.021338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.021236    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:28:32.022060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.022029    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.035234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.035172    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.130177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.130128    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.176104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.176060    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.229610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.229504    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.380456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.380416    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:28:32.380798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.380747    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.502724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.502684    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:32.750662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.750629    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:28:32.961231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:32.961118    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:28:33.007546 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.007488    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:33.009441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.009400    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:33.122252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.122203    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:28:33.125271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.125227    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:28:33.125794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.125763    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/health\": context deadline exceeded"
Nov 01 03:28:33.130317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.129160    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:28:33.131736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.131466    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:33.171963 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.170971    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:28:33.213772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.211057    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:33.213772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.211419    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:33.231416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.231368    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:33.281516 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.279297    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:28:33.365289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.365106    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:28:33.365580 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.365560    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:28:33.366490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.366473    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:28:33.369366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.369349    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:28:33.369854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.369834    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:28:33.370764 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:33.370748    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:34.620352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:34.620313    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:28:34.620784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:34.620602    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy" containerID="cri-o://97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a" gracePeriod=30
Nov 01 03:28:39.008861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.008825    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:28:39.010650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.010621    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e37e1014-0507-4902-a61e-d3f86f0796eb" containerName="revoke-keys"
Nov 01 03:28:39.010843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.010815    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e37e1014-0507-4902-a61e-d3f86f0796eb" containerName="revoke-keys"
Nov 01 03:28:39.011092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.011075    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e37e1014-0507-4902-a61e-d3f86f0796eb" containerName="revoke-keys"
Nov 01 03:28:39.042425 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.042395    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160026    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl5sz\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160083    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160124    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160150    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160287    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160332    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160414    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160475    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.160769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.160521    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.183724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.181900    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:28:39.183724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.182295    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7911e5a0-openshift-default-dockercfg-ltbcl\""
Nov 01 03:28:39.262466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262361    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262434    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262582    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262641    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262683    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262823    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl5sz\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262855    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262889    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262914    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.262961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.262916    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.263450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.263374    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.264185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.264162    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.264951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.264929    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.265490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.265469    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.265739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.265702    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.266587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.266561    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.325312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.325276    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.327050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.326970    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl5sz\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\") pod \"e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") " pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.459830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.459794    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:39.877536 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.877509    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:28:39.939610 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:39.939579    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b7df0b3_0528_4e22_b2f0_3f7ecb7f7b1c.slice/crio-e0036f93fe104eea9bb43db30d32d56b0584cd3d33aa93f271a54216ea948367 WatchSource:0}: Error finding container e0036f93fe104eea9bb43db30d32d56b0584cd3d33aa93f271a54216ea948367: Status 404 returned error can't find the container with id e0036f93fe104eea9bb43db30d32d56b0584cd3d33aa93f271a54216ea948367
Nov 01 03:28:39.939724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.939696    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:28:39.942406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.942372    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:39.942478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.942462    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:39.942542 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.942527    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:39.955420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.955390    4892 generic.go:356] "Generic (PLEG): container finished" podID="32aceb95-8220-4d76-b073-9df54c1906f6" containerID="97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a" exitCode=0
Nov 01 03:28:39.955552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.955452    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"
Nov 01 03:28:39.955552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.955479    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" event={"ID":"32aceb95-8220-4d76-b073-9df54c1906f6","Type":"ContainerDied","Data":"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"}
Nov 01 03:28:39.955683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.955557    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz" event={"ID":"32aceb95-8220-4d76-b073-9df54c1906f6","Type":"ContainerDied","Data":"12810474274dd40d2e5bc7a4e5073bdbaf4bd42a5ecba83f0f6456ab78eaa534"}
Nov 01 03:28:39.955683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.955588    4892 scope.go:122] "RemoveContainer" containerID="97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"
Nov 01 03:28:39.957040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.957017    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerStarted","Data":"e0036f93fe104eea9bb43db30d32d56b0584cd3d33aa93f271a54216ea948367"}
Nov 01 03:28:39.969041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.969012    4892 scope.go:122] "RemoveContainer" containerID="97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"
Nov 01 03:28:39.970456 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:28:39.970423    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a\": container with ID starting with 97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a not found: ID does not exist" containerID="97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"
Nov 01 03:28:39.970538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.970466    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a"} err="failed to get container status \"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a\": rpc error: code = NotFound desc = could not find container \"97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a\": container with ID starting with 97837e76c4f335dea35511f065bd216f16825f952be15f3d56029a818510849a not found: ID does not exist"
Nov 01 03:28:39.982263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982240    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.982377 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982293    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.982434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982375    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.982487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982442    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.982788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982541    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:28:39.982788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982634    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.982788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982740    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:28:39.982788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982765    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.983093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982825    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.983093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982874    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.983093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.982922    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\") pod \"32aceb95-8220-4d76-b073-9df54c1906f6\" (UID: \"32aceb95-8220-4d76-b073-9df54c1906f6\") "
Nov 01 03:28:39.983651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.983626    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:39.983742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.983654    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:39.983818 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.983796    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:28:39.984198 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.984144    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:28:39.986002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.984925    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:28:39.986327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.986299    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:28:39.987415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.987390    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:28:39.987734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.987713    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "kube-api-access-8qdk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:28:39.990253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:39.990224    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo" pod "32aceb95-8220-4d76-b073-9df54c1906f6" (UID: "32aceb95-8220-4d76-b073-9df54c1906f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:28:40.085217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085175    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32aceb95-8220-4d76-b073-9df54c1906f6-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085223    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085237    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32aceb95-8220-4d76-b073-9df54c1906f6-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085252    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085264    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085276    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32aceb95-8220-4d76-b073-9df54c1906f6-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.085663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.085290    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qdk7\" (UniqueName: \"kubernetes.io/projected/32aceb95-8220-4d76-b073-9df54c1906f6-kube-api-access-8qdk7\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:28:40.434058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.434023    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:28:40.440958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.440927    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-qw5tz"]
Nov 01 03:28:40.966142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:40.966108    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerStarted","Data":"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"}
Nov 01 03:28:41.103475 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.103410    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podStartSLOduration=3.103393418 podStartE2EDuration="3.103393418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:41.102351343 +0000 UTC m=+1287.518250868" watchObservedRunningTime="2026-11-01 03:28:41.103393418 +0000 UTC m=+1287.519292947"
Nov 01 03:28:41.460210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.460170    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:41.467139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.467104    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:41.702476 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.702445    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" path="/var/lib/kubelet/pods/32aceb95-8220-4d76-b073-9df54c1906f6/volumes"
Nov 01 03:28:41.971860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.971825    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:41.972944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:41.972921    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:28:42.774800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.774760    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:28:42.775419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.775401    4892 cpu_manager.go:414] "containerMap: removing container" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy"
Nov 01 03:28:42.775419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.775421    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy"
Nov 01 03:28:42.775545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.775535    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="32aceb95-8220-4d76-b073-9df54c1906f6" containerName="istio-proxy"
Nov 01 03:28:42.780074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.780044    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.842793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.842759    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:28:42.939408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939369    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939417    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939534    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939642 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939608    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939638    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7drz\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939807    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939832    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939865    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:42.939954 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:42.939939    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.040685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040587    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.040685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040646    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.040685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040678    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040722    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040810    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.040848    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7drz\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041080    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041308 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041282    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041314    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041491    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041726 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041545    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041726 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041548    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.041851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.041810    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.042310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.042288    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.043706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.043684    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.043890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.043872    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.108888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.108847    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.109070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.108914    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7drz\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.395674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.395559    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:43.447174 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.447140    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:28:43.451314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.451288    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.473886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.473508    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:28:43.478632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.478602    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.522142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.522109    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:28:43.538595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.538523    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:28:43.548877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.548847    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.548880    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7dcs\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.548905    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549076    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxz7\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549113    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549148    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549272 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549249    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549289    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549372    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549419    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549521 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549507    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549568 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549536    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549568 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549558    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549658 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549578    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549725    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549785 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549755    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.549785 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549774    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.549908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.549890    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.651655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651564    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.651655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651605    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.651880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651676    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.651880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651713    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.651880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651737    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.651880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651839    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.652123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651896    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.652123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651924    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7dcs\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.652123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.651958    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.652689 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.652609    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.652909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.652808    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653014    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653033    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxz7\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653093    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653128    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653166    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653201    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653211    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653222    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653284    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653352    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.653524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653437    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653480    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.653890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.653866    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.654167 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.654143    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.654477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.654454    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.654970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.654943    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.655238 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.655215    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.655683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.655380    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.657171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.656045    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.657171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.656324    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.657171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.656377    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.713218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.713185    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.714385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.714360    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxz7\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.719376 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.719353    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7dcs\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.720442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.720417    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\") pod \"e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") " pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.773478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.773445    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:43.797114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.797088    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:43.827029 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.826226    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:28:43.833967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.833928    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:43.834112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.834038    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:43.834112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.834086    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:43.989051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.989013    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" event={"ID":"255a94a8-506a-4c39-b9c3-82299a68e6d6","Type":"ContainerStarted","Data":"c38397b455da33afaeef81923c7cfbcb768260bd26f355d15e1d28fbeea01246"}
Nov 01 03:28:43.989246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:43.989075    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" event={"ID":"255a94a8-506a-4c39-b9c3-82299a68e6d6","Type":"ContainerStarted","Data":"886bcce8d4b69d0f08fe9a1ecd76ccafaad08cdc162f3f2bb5e91dd5f919e7a0"}
Nov 01 03:28:44.113652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.113595    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podStartSLOduration=2.113576834 podStartE2EDuration="2.113576834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:44.110338593 +0000 UTC m=+1290.526238122" watchObservedRunningTime="2026-11-01 03:28:44.113576834 +0000 UTC m=+1290.529476362"
Nov 01 03:28:44.170065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.170036    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:28:44.170593 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:44.170568    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea25d44_8cda_466d_9299_50d80ce9f04f.slice/crio-2153f3384b433fe7b5fd383157ca5bb806cf20d576a0364821f64e5d41ee11b4 WatchSource:0}: Error finding container 2153f3384b433fe7b5fd383157ca5bb806cf20d576a0364821f64e5d41ee11b4: Status 404 returned error can't find the container with id 2153f3384b433fe7b5fd383157ca5bb806cf20d576a0364821f64e5d41ee11b4
Nov 01 03:28:44.174016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.173954    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.174093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.174071    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.174142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.174122    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.221829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.221799    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:28:44.224524 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:44.224494    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f41dfa_56ab_48e0_9ebd_f0d5e220ccf5.slice/crio-e21b48d749697c95bab5e9ebe12d1e78d6f1d99749a456ae3c4ff8b08dd7dd2e WatchSource:0}: Error finding container e21b48d749697c95bab5e9ebe12d1e78d6f1d99749a456ae3c4ff8b08dd7dd2e: Status 404 returned error can't find the container with id e21b48d749697c95bab5e9ebe12d1e78d6f1d99749a456ae3c4ff8b08dd7dd2e
Nov 01 03:28:44.226679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.226641    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.226770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.226716    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.226770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.226751    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:44.396669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.396632    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:44.398365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.398333    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:44.398533 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:44.398385    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:45.016700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.016664    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" event={"ID":"3ea25d44-8cda-466d-9299-50d80ce9f04f","Type":"ContainerStarted","Data":"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"}
Nov 01 03:28:45.017243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.016721    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" event={"ID":"3ea25d44-8cda-466d-9299-50d80ce9f04f","Type":"ContainerStarted","Data":"2153f3384b433fe7b5fd383157ca5bb806cf20d576a0364821f64e5d41ee11b4"}
Nov 01 03:28:45.019233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.019198    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" event={"ID":"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5","Type":"ContainerStarted","Data":"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"}
Nov 01 03:28:45.019391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.019256    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" event={"ID":"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5","Type":"ContainerStarted","Data":"e21b48d749697c95bab5e9ebe12d1e78d6f1d99749a456ae3c4ff8b08dd7dd2e"}
Nov 01 03:28:45.333470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.333348    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podStartSLOduration=2.333327664 podStartE2EDuration="2.333327664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:45.204156132 +0000 UTC m=+1291.620055660" watchObservedRunningTime="2026-11-01 03:28:45.333327664 +0000 UTC m=+1291.749227194"
Nov 01 03:28:45.334079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.334028    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podStartSLOduration=2.334010071 podStartE2EDuration="2.334010071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:45.33242848 +0000 UTC m=+1291.748328010" watchObservedRunningTime="2026-11-01 03:28:45.334010071 +0000 UTC m=+1291.749909602"
Nov 01 03:28:45.411002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.397388    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:45.411002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.397451    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:45.783003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.782504    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:45.788004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.784306    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:45.788004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.784382    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:45.807008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.806214    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:45.809091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.807804    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:45.809091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:45.807879    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:46.399003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.398111    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:46.399003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.398183    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:46.806003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.798831    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:46.806003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.798898    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:46.806003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.804220    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:46.806003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:46.804279    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:47.034009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.029090    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:28:47.092075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.084502    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.129613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.085196    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:28:47.312189 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.310374    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e6e7ea12-openshift-default-dockercfg-vtc4m\""
Nov 01 03:28:47.546549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.538736    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:47.546549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.538844    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560260    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560473    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560600    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560657    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx994\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560725    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560799    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560854    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560907    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.584032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.560943    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.764112    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx994\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.764174    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782067    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782180    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782253    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782365    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782676    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.782894    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.783075    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.789272    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.789543    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.790122    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.790358    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.794300    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.805075    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.805335    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.821590    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.821734    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.827567    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.834749    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.851694    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.851753    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:47.865051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.861946    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.866457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.865682    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.906556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.903753    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx994\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\") pod \"e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") " pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:47.964061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.952872    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:47.964061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:47.954593    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:28:48.055003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.054482    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:48.070634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.069250    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:48.257002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.254072    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:48.415011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.413361    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:48.415011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.413422    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:48.519216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.518442    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:48.774076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.773969    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:48.774699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.774665    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:48.813704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.813591    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:48.813704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.813647    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:48.836103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.836059    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:28:48.843014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.836667    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:28:48.862005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.859093    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:48.862005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.859542    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:28:48.944907 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.942779    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:28:48.967107 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:48.967074    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e865d99_cde7_49f0_824d_c84d3a5d5082.slice/crio-c9d0d6769282b94fd9b0c47f467393eb8ac40ddef0854e71b1c22d1c669ecda4 WatchSource:0}: Error finding container c9d0d6769282b94fd9b0c47f467393eb8ac40ddef0854e71b1c22d1c669ecda4: Status 404 returned error can't find the container with id c9d0d6769282b94fd9b0c47f467393eb8ac40ddef0854e71b1c22d1c669ecda4
Nov 01 03:28:48.996237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.996105    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.996610    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.996672    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.997187    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:48.998609    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.008439    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.009031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.008643    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:28:49.092236 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.092193    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.092888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.092784    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:28:49.093130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.093114    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:28:49.115850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.115597    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:28:49.115850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.115721    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.356027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.355928    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:28:49.416372 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.416250    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:49.416620 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.416584    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:49.458043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.451677    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.886418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.886372    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:49.936099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.934836    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:49.936099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.934899    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:49.936099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.935426    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:49.936099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:49.935481    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:50.110119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.106451    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:28:50.409006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.406908    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerStarted","Data":"c9d0d6769282b94fd9b0c47f467393eb8ac40ddef0854e71b1c22d1c669ecda4"}
Nov 01 03:28:50.413913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.413857    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:50.414186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.414156    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:50.503113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.503070    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:50.503969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.503791    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:50.512827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.512791    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:50.527896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.527851    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:50.817009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.798068    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": context deadline exceeded"
Nov 01 03:28:50.829199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.829157    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:50.897178 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.897142    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:50.897729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.897696    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:50.921337 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.898074    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:50.921337 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.898124    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:50.921337 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.898190    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:50.958821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:50.958790    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:28:51.107408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.107314    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:51.153308 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.150972    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:51.201799 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.201753    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": context deadline exceeded"
Nov 01 03:28:51.204588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.204537    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:51.273406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.273356    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:51.417898 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.417804    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:51.418412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.418379    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:51.418591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.418571    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:28:51.570015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.553586    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerStarted","Data":"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"}
Nov 01 03:28:51.713588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.713486    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podStartSLOduration=5.713467699 podStartE2EDuration="5.713467699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:51.689838519 +0000 UTC m=+1298.105738048" watchObservedRunningTime="2026-11-01 03:28:51.713467699 +0000 UTC m=+1298.129367227"
Nov 01 03:28:51.784077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.783081    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:51.784077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.783147    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:51.794684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.794652    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:28:51.798596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.798565    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:51.798741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:51.798620    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:52.072711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.071606    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:52.078909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.078878    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:52.396368 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.396278    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Nov 01 03:28:52.396635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.396603    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Nov 01 03:28:52.554789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.554757    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:52.556126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.556102    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:28:52.775485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.775440    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Nov 01 03:28:52.775792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.775749    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Nov 01 03:28:52.799338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.799309    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Nov 01 03:28:52.799581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:52.799559    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Nov 01 03:28:53.466531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.466495    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:53.468838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.468816    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:53.470019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.469998    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:28:53.778387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.778359    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:53.778804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.778781    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:53.779578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.779560    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:28:53.803200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.803172    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:53.803501 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.803462    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:53.804229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:53.804212    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:28:57.612866 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:28:57.612762    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:28:57.867382 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.867295    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:28:57.872379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.872356    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.881739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.881706    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.881857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.881795    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.881857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.881836    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.881929 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.881863    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.882089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.882063    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.882199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.882171    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.882199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.882194    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwqb2\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.882315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.882222    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.882315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.882243    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.979696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.979647    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:28:57.984206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984176    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984227    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwqb2\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984275    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984301    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984460    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984505    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984617 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984556    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984617 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984591    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984709    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.984912 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.984891    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.985388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.985365    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.985520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.985484    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.985770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.985746    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.986080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.986059    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.987589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.987567    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:57.987697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:57.987632    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:58.046111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.046078    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwqb2\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:58.046327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.046138    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:58.186363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.186250    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:28:58.624480 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.624453    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:28:58.625127 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:28:58.625102    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod678ba707_c2e0_4a8a_bcb3_33390a9441bd.slice/crio-313d8d4f18d6c03a26f83863ebf08416a2a076a219a485e1204b6bb580c8f75b WatchSource:0}: Error finding container 313d8d4f18d6c03a26f83863ebf08416a2a076a219a485e1204b6bb580c8f75b: Status 404 returned error can't find the container with id 313d8d4f18d6c03a26f83863ebf08416a2a076a219a485e1204b6bb580c8f75b
Nov 01 03:28:58.627812 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.627781    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:58.627885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.627847    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:58.627885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:58.627875    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:28:59.603343 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:59.603301    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" event={"ID":"678ba707-c2e0-4a8a-bcb3-33390a9441bd","Type":"ContainerStarted","Data":"f647feda9d1a75ab8523fc5bb39bca77c23ff46ae9f4040202f849a206fc869e"}
Nov 01 03:28:59.603556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:59.603393    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" event={"ID":"678ba707-c2e0-4a8a-bcb3-33390a9441bd","Type":"ContainerStarted","Data":"313d8d4f18d6c03a26f83863ebf08416a2a076a219a485e1204b6bb580c8f75b"}
Nov 01 03:28:59.831922 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:28:59.831868    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podStartSLOduration=2.831854487 podStartE2EDuration="2.831854487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:28:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:28:59.778265326 +0000 UTC m=+1306.194164837" watchObservedRunningTime="2026-11-01 03:28:59.831854487 +0000 UTC m=+1306.247754079"
Nov 01 03:29:00.212242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:00.212156    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:29:00.214410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:00.214376    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:00.214579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:00.214455    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:01.186696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:01.186655    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:01.187227 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:01.186735    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:02.187294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:02.187260    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:02.187803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:02.187313    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:03.187312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:03.187274    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:03.187839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:03.187351    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:04.186670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:04.186630    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:04.186882 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:04.186702    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:05.187051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:05.186569    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:05.187051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:05.186641    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:06.186622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:06.186580    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Nov 01 03:29:06.186803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:06.186638    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Nov 01 03:29:07.192401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:07.192366    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:29:07.192880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:07.192725    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:29:07.193607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:07.193568    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:29:09.321251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.321199    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.321920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.321860    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.355122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.355078    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.357006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.356954    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.358170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.358142    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.505442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.505399    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.796140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.796086    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:09.828526 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:09.828483    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.370015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.369951    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.481002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.370648    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.484579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.372622    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.485369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.380173    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.501092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.380206    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:29:10.501661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.380226    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:29:10.502589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.380258    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.503417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.380275    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:29:10.504193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.483311    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:29:10.504795 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.504778    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:29:10.518154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.497066    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:10.518154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.508992    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 01 03:29:10.518546 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.518526    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:29:10.520359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.520340    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:29:10.521347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.521331    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:29:10.522119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.522104    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:29:10.651040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.650943    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:29:10.664556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.664528    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:29:10.907952 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.907859    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:29:10.924710 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:10.922106    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:29:11.020139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.020094    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:11.372205 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.372158    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:11.493219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.493155    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.510173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.502631    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.524288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.520758    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.524920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.524876    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.530718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.530687    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:29:11.541015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.538004    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.886006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.885292    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:11.907010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.905269    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:29:11.912007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:11.910108    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:12.312167 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.312131    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:29:12.330417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.327926    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:29:12.352005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.343598    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:29:12.498000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.497823    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:12.543004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.539072    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:12.672728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.672630    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:29:12.953216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.953115    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:12.953868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:12.953844    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:29:13.042932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:13.042904    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:29:13.043482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:13.043455    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:29:13.043588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:13.043517    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:29:27.612157 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:29:27.612119    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:29:28.139874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.139830    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:29:28.146725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.146695    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177480 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177422    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177491    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177617    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177683    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177781    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177854    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.177965 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.177896    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhxll\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.178082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.178017    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.178142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.178096    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.279957    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280044    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhxll\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280121    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280201    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280342    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280448 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280375    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280499 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280456    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280510    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.280609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.280567    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.281057 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.281024    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.282217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.282162    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.282467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.282442    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.282618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.282595    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.282736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.282478    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.285651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.285573    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.286362 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.286312    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.341873 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.341841    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.342101 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.342074    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhxll\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\") pod \"e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") " pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.359114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.359075    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:29:28.464618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.464523    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:28.917916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.917883    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:29:28.919192 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:29:28.919166    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c06d5e6_550f_4454_a6a9_6f5ec557c253.slice/crio-f81dfc4fadc08d65df6461f5c41a27eef687ef53e28f0442c67915972aacf8a7 WatchSource:0}: Error finding container f81dfc4fadc08d65df6461f5c41a27eef687ef53e28f0442c67915972aacf8a7: Status 404 returned error can't find the container with id f81dfc4fadc08d65df6461f5c41a27eef687ef53e28f0442c67915972aacf8a7
Nov 01 03:29:28.922052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.922015    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:29:28.922163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.922085    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:29:28.922163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:28.922117    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:29:29.229224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:29.229138    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" event={"ID":"5c06d5e6-550f-4454-a6a9-6f5ec557c253","Type":"ContainerStarted","Data":"0427b9f32dd3a3946a7297044dfc66d4e3cc84c0247cc7e07108099289484f44"}
Nov 01 03:29:29.229224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:29.229194    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" event={"ID":"5c06d5e6-550f-4454-a6a9-6f5ec557c253","Type":"ContainerStarted","Data":"f81dfc4fadc08d65df6461f5c41a27eef687ef53e28f0442c67915972aacf8a7"}
Nov 01 03:29:29.355226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:29.355170    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podStartSLOduration=1.355155157 podStartE2EDuration="1.355155157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:29:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:29:29.350559203 +0000 UTC m=+1335.766458827" watchObservedRunningTime="2026-11-01 03:29:29.355155157 +0000 UTC m=+1335.771054686"
Nov 01 03:29:29.464931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:29.464888    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:30.467269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:30.467236    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:30.467673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:30.467289    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:31.465330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:31.465291    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:31.465611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:31.465353    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:32.465672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:32.465640    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:32.466105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:32.465692    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:33.464941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:33.464904    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:33.465164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:33.464964    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:34.465405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:34.465373    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:34.465807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:34.465429    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:35.465188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:35.465151    4892 patch_prober.go:29] interesting pod/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Nov 01 03:29:35.465401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:35.465227    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Nov 01 03:29:36.469991 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:36.469943    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:36.470398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:36.470259    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:36.470960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:36.470945    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:29:45.351358 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:45.350153    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:45.542005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:45.541068    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:45.663394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:45.662567    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 01 03:29:45.676147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:45.676075    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:45.923060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:45.922413    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.018005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.017157    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.056146 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.056074    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/health\": context deadline exceeded"
Nov 01 03:29:46.576295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.576230    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Nov 01 03:29:46.588840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.585451    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.638150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.593005    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.638150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.586535    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": context deadline exceeded"
Nov 01 03:29:46.638150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.593284    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:29:46.638150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.593350    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:29:46.638150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.622125    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.724003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.672702    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 01 03:29:46.724003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.721562    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:29:46.749004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.727076    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.749004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.727793    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:29:46.869069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.796123    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.993004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.962090    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:46.993004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:46.962773    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:29:47.104003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.025916    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.252058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.234384    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:29:47.592127 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.592075    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.597008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.593973    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.597675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.597641    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.598157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.598138    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:29:47.631085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.631040    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 01 03:29:47.658493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.631412    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.780902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.736900    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:29:47.780902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.779341    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:29:47.781252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.736952    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded"
Nov 01 03:29:47.794000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.737011    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:29:47.830230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.823073    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:47.962721 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:47.962627    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:48.219467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.219372    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:29:48.219742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.219723    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:29:48.220088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.220072    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:29:48.220241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.220227    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:29:48.235355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.235318    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:29:48.245468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.245443    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:29:48.260569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.259421    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:29:48.279526 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.279499    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:29:48.362463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.362433    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:29:48.399670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.399629    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:29:48.417776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.412410    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:29:48.421024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:29:48.418841    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:29:57.612064 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:29:57.611967    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:30:00.165869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.165833    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:00.170060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.170032    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:00.170235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.170162    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.173904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.173883    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.191563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.191535    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:00.195329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.195311    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.215948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.215911    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:00.220875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.220853    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:00.221006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.220964    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.225708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.225685    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.238230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.238194    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:30:00.276279 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.276244    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:00.282603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.282572    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:00.285727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.285705    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbtj8\" (UniqueName: \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.285906 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.285824    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.285906 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.285866    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.286023 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.285905    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6lvt\" (UniqueName: \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.286113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.286099    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lllfd\" (UniqueName: \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.286232 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.286215    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.318941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.318900    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:00.323727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.323695    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:00.343737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.343708    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:00.387372 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387335    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hrxg\" (UniqueName: \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.387578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387386    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5f8wg\" (UniqueName: \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.387578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387469    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbtj8\" (UniqueName: \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.387705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387615    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.387705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387685    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.387816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387751    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6lvt\" (UniqueName: \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.387816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387808    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.387925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387878    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lllfd\" (UniqueName: \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.387925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.387919    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.388271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.388241    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.388457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.388439    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.388527 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.388505    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.389092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.389068    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.449727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.449648    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6lvt\" (UniqueName: \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\") pod \"maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.449864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.449844    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lllfd\" (UniqueName: \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.449907 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.449852    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbtj8\" (UniqueName: \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\") pod \"maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.488021 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.487951    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:00.489699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.489666    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.489859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.489835    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.490031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.490016    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hrxg\" (UniqueName: \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.490103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.490051    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5f8wg\" (UniqueName: \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.490434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.490408    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.490550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.490525    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.492440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.492104    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:00.510267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.510239    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:00.558050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.553623    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5f8wg\" (UniqueName: \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\") pod \"maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.558050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.555073    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hrxg\" (UniqueName: \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\") pod \"maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.836853 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.836814    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:00.843224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.843193    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:00.881325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.878795    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:00.932671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.932641    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:00.936076 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:00.935463    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23086a2c_3020_490e_8abc_c9a5dec1aac7.slice/crio-3e9d5baecc75aee403b687ff47623e109ce1694e9cd85f3b338cf6aaf5b2c729 WatchSource:0}: Error finding container 3e9d5baecc75aee403b687ff47623e109ce1694e9cd85f3b338cf6aaf5b2c729: Status 404 returned error can't find the container with id 3e9d5baecc75aee403b687ff47623e109ce1694e9cd85f3b338cf6aaf5b2c729
Nov 01 03:30:00.936286 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:00.936093    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod501fef86_5077_4d1d_8f4e_921c4cf51b0d.slice/crio-ec78fb0ad887d95b4f5a65c0353ae1891e49f0f570f322eae2308be2b2757e90 WatchSource:0}: Error finding container ec78fb0ad887d95b4f5a65c0353ae1891e49f0f570f322eae2308be2b2757e90: Status 404 returned error can't find the container with id ec78fb0ad887d95b4f5a65c0353ae1891e49f0f570f322eae2308be2b2757e90
Nov 01 03:30:00.938779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.938754    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:00.988166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.988134    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerStarted","Data":"76beaf06c7fcd8163b3bda7d63534fae40abc1d73740dffea6bdea345a0c767a"}
Nov 01 03:30:00.989786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.989763    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerStarted","Data":"3e9d5baecc75aee403b687ff47623e109ce1694e9cd85f3b338cf6aaf5b2c729"}
Nov 01 03:30:00.992693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:00.992257    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerStarted","Data":"ec78fb0ad887d95b4f5a65c0353ae1891e49f0f570f322eae2308be2b2757e90"}
Nov 01 03:30:01.227224 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:01.227193    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31aface2_b04d_46d2_8d6b_abdf930f2d44.slice/crio-ccab405e5127179c84fe8b9f14bf1ea6c1f4bcb0f839497f7aa9dee9674c0403 WatchSource:0}: Error finding container ccab405e5127179c84fe8b9f14bf1ea6c1f4bcb0f839497f7aa9dee9674c0403: Status 404 returned error can't find the container with id ccab405e5127179c84fe8b9f14bf1ea6c1f4bcb0f839497f7aa9dee9674c0403
Nov 01 03:30:01.227566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:01.227390    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:01.237950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:01.237924    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:01.324032 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:01.323999    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b3a196_664d_4fd5_9d0a_58c1d2048c49.slice/crio-728565691e4b9665dbace2e581b207df060f5346ad179e90b8a2c8654d8241e1 WatchSource:0}: Error finding container 728565691e4b9665dbace2e581b207df060f5346ad179e90b8a2c8654d8241e1: Status 404 returned error can't find the container with id 728565691e4b9665dbace2e581b207df060f5346ad179e90b8a2c8654d8241e1
Nov 01 03:30:02.002728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.002685    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerStarted","Data":"50a6c09976abced41fe740441a3e8e394793645ae8188a23f5db10dd5e27f9ad"}
Nov 01 03:30:02.002908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.002759    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerStarted","Data":"ccab405e5127179c84fe8b9f14bf1ea6c1f4bcb0f839497f7aa9dee9674c0403"}
Nov 01 03:30:02.005020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.004946    4892 generic.go:356] "Generic (PLEG): container finished" podID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerID="18a070a9983269618e2aa4d9a2560dd9759142a78e18aab6992938bef58839ed" exitCode=60
Nov 01 03:30:02.005143 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.005024    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerDied","Data":"18a070a9983269618e2aa4d9a2560dd9759142a78e18aab6992938bef58839ed"}
Nov 01 03:30:02.005384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.005353    4892 scope.go:122] "RemoveContainer" containerID="18a070a9983269618e2aa4d9a2560dd9759142a78e18aab6992938bef58839ed"
Nov 01 03:30:02.006923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.006901    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerStarted","Data":"22c829c868dcc0b42ee3d9e2ebff67a6a7b704f8eab7e5b862c1b9b9f931da8f"}
Nov 01 03:30:02.007064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.006951    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerStarted","Data":"728565691e4b9665dbace2e581b207df060f5346ad179e90b8a2c8654d8241e1"}
Nov 01 03:30:02.008642 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.008612    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerStarted","Data":"3c0ad826c1ae4cdb321c3c27b75b7f2611608cdadc6d64c25762bdd06ca5d12e"}
Nov 01 03:30:02.010171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.010150    4892 generic.go:356] "Generic (PLEG): container finished" podID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerID="41007d05a11647dc088c3ff69b29bf25760289a2d6b361455d2c028c9a6a22b9" exitCode=60
Nov 01 03:30:02.010272 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.010247    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerDied","Data":"41007d05a11647dc088c3ff69b29bf25760289a2d6b361455d2c028c9a6a22b9"}
Nov 01 03:30:02.010462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.010447    4892 scope.go:122] "RemoveContainer" containerID="41007d05a11647dc088c3ff69b29bf25760289a2d6b361455d2c028c9a6a22b9"
Nov 01 03:30:02.120941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.120889    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" podStartSLOduration=2.120875125 podStartE2EDuration="2.120875125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:02.118613887 +0000 UTC m=+1368.534513416" watchObservedRunningTime="2026-11-01 03:30:02.120875125 +0000 UTC m=+1368.536774654"
Nov 01 03:30:02.353522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.353467    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" podStartSLOduration=2.353453664 podStartE2EDuration="2.353453664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:02.352569759 +0000 UTC m=+1368.768469293" watchObservedRunningTime="2026-11-01 03:30:02.353453664 +0000 UTC m=+1368.769353220"
Nov 01 03:30:02.484667 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:02.484612    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" podStartSLOduration=2.484593105 podStartE2EDuration="2.484593105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:02.483354871 +0000 UTC m=+1368.899254396" watchObservedRunningTime="2026-11-01 03:30:02.484593105 +0000 UTC m=+1368.900492633"
Nov 01 03:30:03.018648 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.018605    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerStarted","Data":"c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"}
Nov 01 03:30:03.020298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.020272    4892 generic.go:356] "Generic (PLEG): container finished" podID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerID="22c829c868dcc0b42ee3d9e2ebff67a6a7b704f8eab7e5b862c1b9b9f931da8f" exitCode=60
Nov 01 03:30:03.020421 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.020356    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerDied","Data":"22c829c868dcc0b42ee3d9e2ebff67a6a7b704f8eab7e5b862c1b9b9f931da8f"}
Nov 01 03:30:03.020625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.020610    4892 scope.go:122] "RemoveContainer" containerID="22c829c868dcc0b42ee3d9e2ebff67a6a7b704f8eab7e5b862c1b9b9f931da8f"
Nov 01 03:30:03.022241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.022216    4892 generic.go:356] "Generic (PLEG): container finished" podID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerID="3c0ad826c1ae4cdb321c3c27b75b7f2611608cdadc6d64c25762bdd06ca5d12e" exitCode=60
Nov 01 03:30:03.022366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.022305    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerDied","Data":"3c0ad826c1ae4cdb321c3c27b75b7f2611608cdadc6d64c25762bdd06ca5d12e"}
Nov 01 03:30:03.022947 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.022547    4892 scope.go:122] "RemoveContainer" containerID="3c0ad826c1ae4cdb321c3c27b75b7f2611608cdadc6d64c25762bdd06ca5d12e"
Nov 01 03:30:03.027766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.027741    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerStarted","Data":"5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"}
Nov 01 03:30:03.029687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.029656    4892 generic.go:356] "Generic (PLEG): container finished" podID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerID="50a6c09976abced41fe740441a3e8e394793645ae8188a23f5db10dd5e27f9ad" exitCode=60
Nov 01 03:30:03.029687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.029682    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerDied","Data":"50a6c09976abced41fe740441a3e8e394793645ae8188a23f5db10dd5e27f9ad"}
Nov 01 03:30:03.029946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.029929    4892 scope.go:122] "RemoveContainer" containerID="50a6c09976abced41fe740441a3e8e394793645ae8188a23f5db10dd5e27f9ad"
Nov 01 03:30:03.137291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.137243    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" podStartSLOduration=3.137223149 podStartE2EDuration="3.137223149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:03.133923867 +0000 UTC m=+1369.549823407" watchObservedRunningTime="2026-11-01 03:30:03.137223149 +0000 UTC m=+1369.553123041"
Nov 01 03:30:03.623816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:03.623693    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" podStartSLOduration=3.623655546 podStartE2EDuration="3.623655546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:03.61990721 +0000 UTC m=+1370.035806738" watchObservedRunningTime="2026-11-01 03:30:03.623655546 +0000 UTC m=+1370.039555075"
Nov 01 03:30:04.039814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.039787    4892 generic.go:356] "Generic (PLEG): container finished" podID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerID="0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192" exitCode=60
Nov 01 03:30:04.039957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.039861    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerDied","Data":"0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192"}
Nov 01 03:30:04.039957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.039921    4892 scope.go:122] "RemoveContainer" containerID="50a6c09976abced41fe740441a3e8e394793645ae8188a23f5db10dd5e27f9ad"
Nov 01 03:30:04.040402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.040384    4892 scope.go:122] "RemoveContainer" containerID="0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192"
Nov 01 03:30:04.040791 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:04.040769    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf_odh-ai-gateway-infra(31aface2-b04d-46d2-8d6b-abdf930f2d44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44"
Nov 01 03:30:04.042708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.042686    4892 generic.go:356] "Generic (PLEG): container finished" podID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerID="c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4" exitCode=60
Nov 01 03:30:04.042800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.042762    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerDied","Data":"c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"}
Nov 01 03:30:04.043183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.043163    4892 scope.go:122] "RemoveContainer" containerID="c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"
Nov 01 03:30:04.043575 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:04.043551    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529_odh-ai-gateway-infra(6baa6489-87d1-43dd-afe8-278a1e4a98a9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9"
Nov 01 03:30:04.044700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.044679    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerStarted","Data":"0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"}
Nov 01 03:30:04.044949 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.044936    4892 scope.go:122] "RemoveContainer" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"
Nov 01 03:30:04.045282 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:04.045259    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp_odh-ai-gateway-infra(e3b3a196-664d-4fd5-9d0a-58c1d2048c49)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49"
Nov 01 03:30:04.046732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.046711    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerStarted","Data":"35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"}
Nov 01 03:30:04.047075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.047055    4892 scope.go:122] "RemoveContainer" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"
Nov 01 03:30:04.047420 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:04.047396    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg_odh-ai-gateway-infra(23086a2c-3020-490e-8abc-c9a5dec1aac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7"
Nov 01 03:30:04.048508 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.048490    4892 generic.go:356] "Generic (PLEG): container finished" podID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerID="5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799" exitCode=60
Nov 01 03:30:04.048600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.048567    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerDied","Data":"5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"}
Nov 01 03:30:04.048754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.048741    4892 scope.go:122] "RemoveContainer" containerID="5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"
Nov 01 03:30:04.049015 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:04.048998    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k_odh-ai-gateway-infra(501fef86-5077-4d1d-8f4e-921c4cf51b0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d"
Nov 01 03:30:04.200357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.200329    4892 scope.go:122] "RemoveContainer" containerID="18a070a9983269618e2aa4d9a2560dd9759142a78e18aab6992938bef58839ed"
Nov 01 03:30:04.213936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:04.213916    4892 scope.go:122] "RemoveContainer" containerID="41007d05a11647dc088c3ff69b29bf25760289a2d6b361455d2c028c9a6a22b9"
Nov 01 03:30:05.057482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.057454    4892 scope.go:122] "RemoveContainer" containerID="0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192"
Nov 01 03:30:05.057948 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:05.057829    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf_odh-ai-gateway-infra(31aface2-b04d-46d2-8d6b-abdf930f2d44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44"
Nov 01 03:30:05.059050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.059023    4892 scope.go:122] "RemoveContainer" containerID="c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"
Nov 01 03:30:05.059391 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:05.059369    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529_odh-ai-gateway-infra(6baa6489-87d1-43dd-afe8-278a1e4a98a9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9"
Nov 01 03:30:05.060657 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.060634    4892 generic.go:356] "Generic (PLEG): container finished" podID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470" exitCode=60
Nov 01 03:30:05.060761 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.060721    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerDied","Data":"0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"}
Nov 01 03:30:05.060824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.060774    4892 scope.go:122] "RemoveContainer" containerID="22c829c868dcc0b42ee3d9e2ebff67a6a7b704f8eab7e5b862c1b9b9f931da8f"
Nov 01 03:30:05.061046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.061025    4892 scope.go:122] "RemoveContainer" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"
Nov 01 03:30:05.061343 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:05.061318    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp_odh-ai-gateway-infra(e3b3a196-664d-4fd5-9d0a-58c1d2048c49)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49"
Nov 01 03:30:05.062591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.062571    4892 generic.go:356] "Generic (PLEG): container finished" podID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0" exitCode=60
Nov 01 03:30:05.062679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.062655    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerDied","Data":"35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"}
Nov 01 03:30:05.062906 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.062886    4892 scope.go:122] "RemoveContainer" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"
Nov 01 03:30:05.063220 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:05.063198    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg_odh-ai-gateway-infra(23086a2c-3020-490e-8abc-c9a5dec1aac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7"
Nov 01 03:30:05.064447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.064431    4892 scope.go:122] "RemoveContainer" containerID="5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"
Nov 01 03:30:05.064675 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:05.064659    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k_odh-ai-gateway-infra(501fef86-5077-4d1d-8f4e-921c4cf51b0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d"
Nov 01 03:30:05.074389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:05.074375    4892 scope.go:122] "RemoveContainer" containerID="3c0ad826c1ae4cdb321c3c27b75b7f2611608cdadc6d64c25762bdd06ca5d12e"
Nov 01 03:30:06.073128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:06.073096    4892 scope.go:122] "RemoveContainer" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"
Nov 01 03:30:06.073541 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:06.073477    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp_odh-ai-gateway-infra(e3b3a196-664d-4fd5-9d0a-58c1d2048c49)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49"
Nov 01 03:30:06.074651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:06.074632    4892 scope.go:122] "RemoveContainer" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"
Nov 01 03:30:06.074990 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:06.074958    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg_odh-ai-gateway-infra(23086a2c-3020-490e-8abc-c9a5dec1aac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7"
Nov 01 03:30:15.694073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:15.694033    4892 scope.go:122] "RemoveContainer" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"
Nov 01 03:30:16.159242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:16.159199    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerStarted","Data":"1ac058abfca8f01f2d31f1880d4e8ba08711b02e3adfaf36070647eb71bf105f"}
Nov 01 03:30:16.694529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:16.694497    4892 scope.go:122] "RemoveContainer" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"
Nov 01 03:30:16.694908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:16.694665    4892 scope.go:122] "RemoveContainer" containerID="0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192"
Nov 01 03:30:16.694908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:16.694799    4892 scope.go:122] "RemoveContainer" containerID="5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"
Nov 01 03:30:17.168128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.168094    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerStarted","Data":"c063dc8013c508101d8bcdc282035bc4f462b25515eb50866ec6b09ec6d232a3"}
Nov 01 03:30:17.170095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.170069    4892 generic.go:356] "Generic (PLEG): container finished" podID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerID="1ac058abfca8f01f2d31f1880d4e8ba08711b02e3adfaf36070647eb71bf105f" exitCode=60
Nov 01 03:30:17.170263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.170143    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerDied","Data":"1ac058abfca8f01f2d31f1880d4e8ba08711b02e3adfaf36070647eb71bf105f"}
Nov 01 03:30:17.170263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.170202    4892 scope.go:122] "RemoveContainer" containerID="35c25b624f0d6b752a55cbda262dd7494261fc51baeca363b2142ffb765e71a0"
Nov 01 03:30:17.170626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.170612    4892 scope.go:122] "RemoveContainer" containerID="1ac058abfca8f01f2d31f1880d4e8ba08711b02e3adfaf36070647eb71bf105f"
Nov 01 03:30:17.170904 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:17.170884    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg_odh-ai-gateway-infra(23086a2c-3020-490e-8abc-c9a5dec1aac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7"
Nov 01 03:30:17.291523 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.291490    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:17.694673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:17.694636    4892 scope.go:122] "RemoveContainer" containerID="c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"
Nov 01 03:30:18.179732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.179648    4892 generic.go:356] "Generic (PLEG): container finished" podID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerID="c063dc8013c508101d8bcdc282035bc4f462b25515eb50866ec6b09ec6d232a3" exitCode=60
Nov 01 03:30:18.179732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.179720    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerDied","Data":"c063dc8013c508101d8bcdc282035bc4f462b25515eb50866ec6b09ec6d232a3"}
Nov 01 03:30:18.179966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.179774    4892 scope.go:122] "RemoveContainer" containerID="0ecc95a5b0a8ae40965ca2a31b5fa6b12eb972621e82e0ac6eaa1780fd311192"
Nov 01 03:30:18.180265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.180245    4892 scope.go:122] "RemoveContainer" containerID="c063dc8013c508101d8bcdc282035bc4f462b25515eb50866ec6b09ec6d232a3"
Nov 01 03:30:18.180735 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:18.180648    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf_odh-ai-gateway-infra(31aface2-b04d-46d2-8d6b-abdf930f2d44)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44"
Nov 01 03:30:18.181911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.181887    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerStarted","Data":"d1bdbd79981e1219b9f6e28f82b7b1d4374c173aff799ecd215f61c702e69b25"}
Nov 01 03:30:18.183796 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.183779    4892 generic.go:356] "Generic (PLEG): container finished" podID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerID="56e7974709de9742bb0230df6a395219db67c14f073e70a045336823078cebe4" exitCode=60
Nov 01 03:30:18.183883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.183850    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerDied","Data":"56e7974709de9742bb0230df6a395219db67c14f073e70a045336823078cebe4"}
Nov 01 03:30:18.184253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.184236    4892 scope.go:122] "RemoveContainer" containerID="56e7974709de9742bb0230df6a395219db67c14f073e70a045336823078cebe4"
Nov 01 03:30:18.184579 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:18.184552    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp_odh-ai-gateway-infra(e3b3a196-664d-4fd5-9d0a-58c1d2048c49)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49"
Nov 01 03:30:18.187493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.187466    4892 generic.go:356] "Generic (PLEG): container finished" podID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerID="11ccffc3265edb77638de2826953ae43588d4bf1bf75e950cd9a767648791b87" exitCode=60
Nov 01 03:30:18.187581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.187537    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerDied","Data":"11ccffc3265edb77638de2826953ae43588d4bf1bf75e950cd9a767648791b87"}
Nov 01 03:30:18.187876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.187848    4892 scope.go:122] "RemoveContainer" containerID="11ccffc3265edb77638de2826953ae43588d4bf1bf75e950cd9a767648791b87"
Nov 01 03:30:18.188162 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:18.188143    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k_odh-ai-gateway-infra(501fef86-5077-4d1d-8f4e-921c4cf51b0d)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d"
Nov 01 03:30:18.193522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.193498    4892 scope.go:122] "RemoveContainer" containerID="0e61eac7b03b64cffa48473e3502ad1ebbfd6273c769188ec201afc0dbec5470"
Nov 01 03:30:18.208148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.208124    4892 scope.go:122] "RemoveContainer" containerID="5530c8afcaf7fba3a00d4e65e3528323a272bdde3d04b3909bbad7ae1e798799"
Nov 01 03:30:18.307618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.307585    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:18.370643 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.370610    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:18.491278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.491195    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\") pod \"23086a2c-3020-490e-8abc-c9a5dec1aac7\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") "
Nov 01 03:30:18.491432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.491281    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\" (UniqueName: \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\") pod \"23086a2c-3020-490e-8abc-c9a5dec1aac7\" (UID: \"23086a2c-3020-490e-8abc-c9a5dec1aac7\") "
Nov 01 03:30:18.491639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.491608    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca" pod "23086a2c-3020-490e-8abc-c9a5dec1aac7" (UID: "23086a2c-3020-490e-8abc-c9a5dec1aac7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:30:18.492108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.492090    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/23086a2c-3020-490e-8abc-c9a5dec1aac7-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:18.493502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.493467    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd" pod "23086a2c-3020-490e-8abc-c9a5dec1aac7" (UID: "23086a2c-3020-490e-8abc-c9a5dec1aac7"). InnerVolumeSpecName "kube-api-access-lllfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:30:18.593724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:18.593684    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lllfd\" (UniqueName: \"kubernetes.io/projected/23086a2c-3020-490e-8abc-c9a5dec1aac7-kube-api-access-lllfd\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:19.198194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.198164    4892 generic.go:356] "Generic (PLEG): container finished" podID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerID="d1bdbd79981e1219b9f6e28f82b7b1d4374c173aff799ecd215f61c702e69b25" exitCode=60
Nov 01 03:30:19.198598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.198238    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerDied","Data":"d1bdbd79981e1219b9f6e28f82b7b1d4374c173aff799ecd215f61c702e69b25"}
Nov 01 03:30:19.198598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.198320    4892 scope.go:122] "RemoveContainer" containerID="c73254a7b488cfd7114d1b7fb55ae1594a20cb6a3a7273b6cceb70ead7f547a4"
Nov 01 03:30:19.198784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.198761    4892 scope.go:122] "RemoveContainer" containerID="d1bdbd79981e1219b9f6e28f82b7b1d4374c173aff799ecd215f61c702e69b25"
Nov 01 03:30:19.199737 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:19.199709    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529_odh-ai-gateway-infra(6baa6489-87d1-43dd-afe8-278a1e4a98a9)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9"
Nov 01 03:30:19.205613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.205584    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg" event={"ID":"23086a2c-3020-490e-8abc-c9a5dec1aac7","Type":"ContainerDied","Data":"3e9d5baecc75aee403b687ff47623e109ce1694e9cd85f3b338cf6aaf5b2c729"}
Nov 01 03:30:19.205849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.205586    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"
Nov 01 03:30:19.216733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.216699    4892 scope.go:122] "RemoveContainer" containerID="1ac058abfca8f01f2d31f1880d4e8ba08711b02e3adfaf36070647eb71bf105f"
Nov 01 03:30:19.382681 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.382649    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:19.407420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.407383    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\") pod \"31aface2-b04d-46d2-8d6b-abdf930f2d44\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") "
Nov 01 03:30:19.407589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.407478    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\" (UniqueName: \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\") pod \"31aface2-b04d-46d2-8d6b-abdf930f2d44\" (UID: \"31aface2-b04d-46d2-8d6b-abdf930f2d44\") "
Nov 01 03:30:19.407718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.407694    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca" pod "31aface2-b04d-46d2-8d6b-abdf930f2d44" (UID: "31aface2-b04d-46d2-8d6b-abdf930f2d44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:30:19.408093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.408068    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/31aface2-b04d-46d2-8d6b-abdf930f2d44-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:19.409537 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.409518    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg" pod "31aface2-b04d-46d2-8d6b-abdf930f2d44" (UID: "31aface2-b04d-46d2-8d6b-abdf930f2d44"). InnerVolumeSpecName "kube-api-access-2hrxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:30:19.425148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.425118    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:19.512849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.512810    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hrxg\" (UniqueName: \"kubernetes.io/projected/31aface2-b04d-46d2-8d6b-abdf930f2d44-kube-api-access-2hrxg\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:19.544184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.543774    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:19.549639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.549607    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:19.554147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.554117    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891730-7xbsg"]
Nov 01 03:30:19.587625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.587596    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:19.613586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.613554    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\" (UniqueName: \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\") pod \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") "
Nov 01 03:30:19.613767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.613727    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\") pod \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\" (UID: \"501fef86-5077-4d1d-8f4e-921c4cf51b0d\") "
Nov 01 03:30:19.614116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.614090    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca" pod "501fef86-5077-4d1d-8f4e-921c4cf51b0d" (UID: "501fef86-5077-4d1d-8f4e-921c4cf51b0d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:30:19.614357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.614342    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/501fef86-5077-4d1d-8f4e-921c4cf51b0d-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:19.615845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.615823    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8" pod "501fef86-5077-4d1d-8f4e-921c4cf51b0d" (UID: "501fef86-5077-4d1d-8f4e-921c4cf51b0d"). InnerVolumeSpecName "kube-api-access-pbtj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:30:19.705262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.705226    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" path="/var/lib/kubelet/pods/23086a2c-3020-490e-8abc-c9a5dec1aac7/volumes"
Nov 01 03:30:19.715693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.715661    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbtj8\" (UniqueName: \"kubernetes.io/projected/501fef86-5077-4d1d-8f4e-921c4cf51b0d-kube-api-access-pbtj8\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:19.773677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.773647    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:19.959703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:19.959672    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:20.019489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.019456    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\" (UniqueName: \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\") pod \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") "
Nov 01 03:30:20.019788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.019764    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\") pod \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\" (UID: \"e3b3a196-664d-4fd5-9d0a-58c1d2048c49\") "
Nov 01 03:30:20.020188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.020160    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca" pod "e3b3a196-664d-4fd5-9d0a-58c1d2048c49" (UID: "e3b3a196-664d-4fd5-9d0a-58c1d2048c49"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:30:20.020464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.020446    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:20.021647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.021623    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg" pod "e3b3a196-664d-4fd5-9d0a-58c1d2048c49" (UID: "e3b3a196-664d-4fd5-9d0a-58c1d2048c49"). InnerVolumeSpecName "kube-api-access-5f8wg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:30:20.122082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.122046    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5f8wg\" (UniqueName: \"kubernetes.io/projected/e3b3a196-664d-4fd5-9d0a-58c1d2048c49-kube-api-access-5f8wg\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:20.220216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.220180    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"
Nov 01 03:30:20.220216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.220193    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf" event={"ID":"31aface2-b04d-46d2-8d6b-abdf930f2d44","Type":"ContainerDied","Data":"ccab405e5127179c84fe8b9f14bf1ea6c1f4bcb0f839497f7aa9dee9674c0403"}
Nov 01 03:30:20.220777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.220259    4892 scope.go:122] "RemoveContainer" containerID="c063dc8013c508101d8bcdc282035bc4f462b25515eb50866ec6b09ec6d232a3"
Nov 01 03:30:20.225357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.225336    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"
Nov 01 03:30:20.225834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.225374    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp" event={"ID":"e3b3a196-664d-4fd5-9d0a-58c1d2048c49","Type":"ContainerDied","Data":"728565691e4b9665dbace2e581b207df060f5346ad179e90b8a2c8654d8241e1"}
Nov 01 03:30:20.228470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.228443    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k" event={"ID":"501fef86-5077-4d1d-8f4e-921c4cf51b0d","Type":"ContainerDied","Data":"ec78fb0ad887d95b4f5a65c0353ae1891e49f0f570f322eae2308be2b2757e90"}
Nov 01 03:30:20.228470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.228461    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"
Nov 01 03:30:20.232209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.232093    4892 scope.go:122] "RemoveContainer" containerID="56e7974709de9742bb0230df6a395219db67c14f073e70a045336823078cebe4"
Nov 01 03:30:20.242504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.242476    4892 scope.go:122] "RemoveContainer" containerID="11ccffc3265edb77638de2826953ae43588d4bf1bf75e950cd9a767648791b87"
Nov 01 03:30:20.389294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.389262    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:20.393765 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.393738    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-d5c42938-29891730-5qqlf"]
Nov 01 03:30:20.394100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.394081    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:20.427272 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.427241    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\") pod \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") "
Nov 01 03:30:20.427464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.427343    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\" (UniqueName: \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\") pod \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\" (UID: \"6baa6489-87d1-43dd-afe8-278a1e4a98a9\") "
Nov 01 03:30:20.427665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.427641    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca" pod "6baa6489-87d1-43dd-afe8-278a1e4a98a9" (UID: "6baa6489-87d1-43dd-afe8-278a1e4a98a9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:30:20.427916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.427892    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6baa6489-87d1-43dd-afe8-278a1e4a98a9-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:20.429584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.429559    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt" pod "6baa6489-87d1-43dd-afe8-278a1e4a98a9" (UID: "6baa6489-87d1-43dd-afe8-278a1e4a98a9"). InnerVolumeSpecName "kube-api-access-f6lvt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:30:20.529333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.529293    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6lvt\" (UniqueName: \"kubernetes.io/projected/6baa6489-87d1-43dd-afe8-278a1e4a98a9-kube-api-access-f6lvt\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:30:20.625489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.625157    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:20.640000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.637849    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-e6e7ea12-29891730-k6ggp"]
Nov 01 03:30:20.739050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.739015    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:20.743119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:20.743082    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-19faa5-29891730-nkr9k"]
Nov 01 03:30:21.237885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.237795    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"
Nov 01 03:30:21.237885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.237806    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529" event={"ID":"6baa6489-87d1-43dd-afe8-278a1e4a98a9","Type":"ContainerDied","Data":"76beaf06c7fcd8163b3bda7d63534fae40abc1d73740dffea6bdea345a0c767a"}
Nov 01 03:30:21.238432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.237887    4892 scope.go:122] "RemoveContainer" containerID="d1bdbd79981e1219b9f6e28f82b7b1d4374c173aff799ecd215f61c702e69b25"
Nov 01 03:30:21.406816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.406771    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:21.410659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.410629    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w5-c22dd8-29891730-bc529"]
Nov 01 03:30:21.706135 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.706099    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" path="/var/lib/kubelet/pods/31aface2-b04d-46d2-8d6b-abdf930f2d44/volumes"
Nov 01 03:30:21.706667 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.706648    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" path="/var/lib/kubelet/pods/501fef86-5077-4d1d-8f4e-921c4cf51b0d/volumes"
Nov 01 03:30:21.707111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.707096    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" path="/var/lib/kubelet/pods/6baa6489-87d1-43dd-afe8-278a1e4a98a9/volumes"
Nov 01 03:30:21.708063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:21.708043    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" path="/var/lib/kubelet/pods/e3b3a196-664d-4fd5-9d0a-58c1d2048c49/volumes"
Nov 01 03:30:27.615482 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:27.615405    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:30:35.660899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.660866    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:30:35.661495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661479    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661497    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661513    4892 cpu_manager.go:414] "containerMap: removing container" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661518    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661524    4892 cpu_manager.go:414] "containerMap: removing container" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661529    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.661539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661538    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661544    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661560    4892 cpu_manager.go:414] "containerMap: removing container" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661565    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661572    4892 cpu_manager.go:414] "containerMap: removing container" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661577    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661584    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661588    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661598    4892 cpu_manager.go:414] "containerMap: removing container" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661602    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661615    4892 cpu_manager.go:414] "containerMap: removing container" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661620    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661629    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661633    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661639    4892 cpu_manager.go:414] "containerMap: removing container" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.661722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661644    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661731    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661738    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661744    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661752    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661759    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661765    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661772    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661782    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661792    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661801    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661809    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.662172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.661814    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="31aface2-b04d-46d2-8d6b-abdf930f2d44" containerName="cleanup"
Nov 01 03:30:35.665628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.665606    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.702477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.702450    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:30:35.703045 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703030    4892 cpu_manager.go:414] "containerMap: removing container" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.703100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703047    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.703100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703060    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.703100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703065    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.703100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703091    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.703100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703096    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.703253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703114    4892 cpu_manager.go:414] "containerMap: removing container" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.703253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703119    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="501fef86-5077-4d1d-8f4e-921c4cf51b0d" containerName="cleanup"
Nov 01 03:30:35.703253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703215    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3b3a196-664d-4fd5-9d0a-58c1d2048c49" containerName="cleanup"
Nov 01 03:30:35.703253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703227    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="23086a2c-3020-490e-8abc-c9a5dec1aac7" containerName="cleanup"
Nov 01 03:30:35.703253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.703233    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6baa6489-87d1-43dd-afe8-278a1e4a98a9" containerName="cleanup"
Nov 01 03:30:35.706753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.706720    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.729473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.729444    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:30:35.789932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.789877    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919086    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919135    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919194    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919220    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919259    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919290    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919344    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919374    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919441    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr856\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919542    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919576    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919666    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919716    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl2gg\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919749    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919797    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919855    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919893    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:35.920064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:35.919933    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.023088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023048    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.023384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023360    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl2gg\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.023513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023498    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.023636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023623    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.023770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023755    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.023880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023866    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.024008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.023974    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.024158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024144    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.024269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024253    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.024400 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024387    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.024488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024476    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.024596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024580    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.024696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024682    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.024827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024815    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.024924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.024911    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.025097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.025080    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr856\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.025306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.025292    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.025424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.025411    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.028547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.028515    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.029105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.029075    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.030039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.029424    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.030039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.029446    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.030324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.030295    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.031233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.030496    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.031233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.030566    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.031233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.030829    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.031233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.031027    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.031233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.031206    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.031668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.031645    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.032027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.031960    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.033305 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.033282    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.033951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.033928    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.093998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.093951    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl2gg\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.094158 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.094091    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-j2sgp\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.094466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.094445    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.106563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.106529    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr856\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-pfv7l\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.279579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.279538    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:36.321086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.321050    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:36.731478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.731447    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:30:36.733269 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:36.733238    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ec406b_0bb7_426e_8e3a_c7596539bb8c.slice/crio-32bf95b87c9b76db461c5145882c9360b7273bb95a99c5528e2faf7cd845856d WatchSource:0}: Error finding container 32bf95b87c9b76db461c5145882c9360b7273bb95a99c5528e2faf7cd845856d: Status 404 returned error can't find the container with id 32bf95b87c9b76db461c5145882c9360b7273bb95a99c5528e2faf7cd845856d
Nov 01 03:30:36.736089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.736052    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:36.736183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.736128    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:36.736183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.736160    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:36.789588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.789557    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:30:36.794020 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:30:36.793903    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode04085b7_baf7_445d_93e2_491f6810ccba.slice/crio-6c13ab80fc06597f5ecaa33061e4e67793a65d2d72b1b110d718ec437d18b0ba WatchSource:0}: Error finding container 6c13ab80fc06597f5ecaa33061e4e67793a65d2d72b1b110d718ec437d18b0ba: Status 404 returned error can't find the container with id 6c13ab80fc06597f5ecaa33061e4e67793a65d2d72b1b110d718ec437d18b0ba
Nov 01 03:30:36.796937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.796899    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:36.797079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.797000    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:36.797079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:36.797042    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:37.404086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.404052    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" event={"ID":"e04085b7-baf7-445d-93e2-491f6810ccba","Type":"ContainerStarted","Data":"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"}
Nov 01 03:30:37.404300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.404117    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" event={"ID":"e04085b7-baf7-445d-93e2-491f6810ccba","Type":"ContainerStarted","Data":"6c13ab80fc06597f5ecaa33061e4e67793a65d2d72b1b110d718ec437d18b0ba"}
Nov 01 03:30:37.405940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.405907    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" event={"ID":"90ec406b-0bb7-426e-8e3a-c7596539bb8c","Type":"ContainerStarted","Data":"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"}
Nov 01 03:30:37.406129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.405963    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" event={"ID":"90ec406b-0bb7-426e-8e3a-c7596539bb8c","Type":"ContainerStarted","Data":"32bf95b87c9b76db461c5145882c9360b7273bb95a99c5528e2faf7cd845856d"}
Nov 01 03:30:37.554007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.550572    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podStartSLOduration=2.550547797 podStartE2EDuration="2.550547797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:37.527491685 +0000 UTC m=+1403.943391216" watchObservedRunningTime="2026-11-01 03:30:37.550547797 +0000 UTC m=+1403.966447327"
Nov 01 03:30:37.691904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:37.691797    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podStartSLOduration=2.691780385 podStartE2EDuration="2.691780385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:30:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:30:37.690016235 +0000 UTC m=+1404.105915763" watchObservedRunningTime="2026-11-01 03:30:37.691780385 +0000 UTC m=+1404.107679914"
Nov 01 03:30:38.283004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.280156    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:38.283004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.282457    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:38.283004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.282560    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:38.326002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.325498    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:38.351006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.341628    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:38.351006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:38.341700    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:39.218004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.212295    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:30:39.243009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.241078    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:39.252005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.250073    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:39.319004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.311519    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:39.319004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.311598    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:39.354005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.338379    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:39.354005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.338552    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:39.418007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.414292    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:30:39.462298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.462209    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:39.515064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.511082    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded"
Nov 01 03:30:39.806261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.802971    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:39.988003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.987360    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.006250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:39.994072    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/health\": context deadline exceeded"
Nov 01 03:30:40.250782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.250666    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.284125 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.261130    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:30:40.303188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.303146    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:40.303506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.303471    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:40.304458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.304426    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.310254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.310228    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:30:40.331347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.314604    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.332378 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.332342    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:30:40.339517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.339486    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:40.339794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.339745    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:40.366110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.366062    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.417007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.415078    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded"
Nov 01 03:30:40.417007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.415562    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:30:40.478005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.468571    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.478005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.469012    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:30:40.502069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.488290    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/health\": context deadline exceeded"
Nov 01 03:30:40.577577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.576198    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.577577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.576612    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:30:40.627708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.619386    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.814133 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.814078    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:40.818001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:40.814834    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:30:41.013348 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.013302    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.031001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.014002    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:30:41.060103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.060056    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.193059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.176394    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:30:41.279014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.253921    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.330010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.325092    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:30:41.330010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.325092    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.381521 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.381481    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:41.405366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.405320    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:41.405620 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.382649    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:41.405737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.405717    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:41.415352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.415306    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.545062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.539080    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.607055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.593109    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.664004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.652483    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.664004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.652897    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:30:41.831270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.831079    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-b6kpb" podUID="a7b525a3-688f-4c08-8b4d-527dcbf2d359" containerName="cert-manager-webhook" probeResult="failure" output="Get \"http://10.134.0.17:6080/healthz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:41.974052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:41.970393    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:30:42.032005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.021602    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.263006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.259204    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.291003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.290602    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:42.291003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.290659    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:42.316005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.291811    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.345005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.325399    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:42.345005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.325466    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:42.361006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.360315    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.361006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.360716    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.453005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.452200    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.486006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.460399    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.543228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.543102    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:42.601591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.601560    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:30:42.612668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.608449    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:30:42.614663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.614638    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:30:42.653655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:42.653611    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.156005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.070383    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.279012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.266042    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.310681 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.310651    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:30:43.312563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.312534    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:43.333507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.333423    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:43.336777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.336748    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:43.337030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.337002    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:43.434515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.434413    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": context deadline exceeded"
Nov 01 03:30:43.478042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.467098    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.500366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.500321    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.525051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.525008    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:30:43.679217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.679173    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:43.786935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:43.727095    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.47:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.124196 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.124089    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": context deadline exceeded"
Nov 01 03:30:44.139014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.128437    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/health\": context deadline exceeded"
Nov 01 03:30:44.271613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.271564    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.316562 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.316505    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.317036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.317002    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" probeResult="failure" output="Get \"https://10.134.0.45:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.390064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.385590    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:44.390064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.385654    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:44.444999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.444004    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:30:44.448998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.445571    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.488003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.462480    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:44.488003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.462544    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:44.505004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.501143    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.545134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.545034    4892 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.616252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.616222    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:30:44.707265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.707178    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:30:44.847135 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.847074    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" probeResult="failure" output="Get \"https://10.134.0.46:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:44.858016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.853359    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:30:44.907851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:44.906897    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:30:45.118278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:45.117232    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:30:45.287354 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:45.287296    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:45.287631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:45.287598    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:45.369008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:45.368868    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:45.369282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:45.369253    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:46.281218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:46.281165    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Nov 01 03:30:46.281774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:46.281232    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Nov 01 03:30:46.322489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:46.322448    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Nov 01 03:30:46.322774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:46.322741    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Nov 01 03:30:48.328730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.327962    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:30:48.328730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.328060    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:48.364229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.364109    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:30:48.364865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.364182    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:30:48.714102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.713785    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:48.715146 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.714930    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:48.718239 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.718140    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:30:48.796024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.794368    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:48.796024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.795787    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:48.796899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:48.796842    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:30:52.555774 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.555730    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:52.556300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.555809    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:52.727601 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.727571    4892 generic.go:356] "Generic (PLEG): container finished" podID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerID="743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801" exitCode=0
Nov 01 03:30:52.727765 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.727644    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerDied","Data":"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"}
Nov 01 03:30:52.728170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.728151    4892 scope.go:122] "RemoveContainer" containerID="743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"
Nov 01 03:30:52.729516 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.729493    4892 generic.go:356] "Generic (PLEG): container finished" podID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerID="c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc" exitCode=0
Nov 01 03:30:52.729595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.729563    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerDied","Data":"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"}
Nov 01 03:30:52.729914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.729887    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:52.729954 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.729927    4892 scope.go:122] "RemoveContainer" containerID="c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"
Nov 01 03:30:52.730011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.729957    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:52.730011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.730003    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:52.731359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.731334    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:52.731429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.731397    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:52.731469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:52.731432    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:30:54.046888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.046849    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerStarted","Data":"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"}
Nov 01 03:30:54.051047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.051012    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerStarted","Data":"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"}
Nov 01 03:30:54.092830 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.092785    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:30:54.097151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.097114    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:54.097831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.097772    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:54.466130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.466040    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:30:54.472042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.469165    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:54.472042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:54.469240    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:30:55.099091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:55.099050    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:55.131325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:55.131266    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:55.462062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:55.461433    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:55.462062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:55.461514    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:30:56.072395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:56.072361    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:56.072616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:56.072422    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:56.462567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:56.462470    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:56.462567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:56.462538    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:30:56.974401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:56.973252    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:30:57.071412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:57.071371    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:57.071705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:57.071675    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:57.465084 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:57.465047    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:57.465739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:57.465704    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:30:57.616638 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:30:57.616596    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:30:58.072005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:58.071347    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:58.072005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:58.071399    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:58.465998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:58.465380    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:58.465998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:58.465448    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:30:59.073528 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:59.073348    4892 patch_prober.go:29] interesting pod/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Nov 01 03:30:59.073528 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:59.073412    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Nov 01 03:30:59.461054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:59.460952    4892 patch_prober.go:29] interesting pod/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Nov 01 03:30:59.461054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:30:59.461026    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Nov 01 03:31:00.076928 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:00.076897    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:31:00.077773 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:00.077756    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:31:00.078711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:00.078686    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:31:00.464954 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:00.464866    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:31:00.465955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:00.465928    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:31:27.613262 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:27.613166    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:31:33.925665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:33.925625    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:31:33.928778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:33.928761    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:33.981810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:33.981767    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d262c155-gw-openshift-default-dockercfg-n57vz\""
Nov 01 03:31:34.037249 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037209    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037394    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037436    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bsb5\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037496    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037548    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037588    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037735    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037769    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.037789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037771    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:31:34.038023 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.037995    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.140163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.140118    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.140457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.140434    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bsb5\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.140595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.140581    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.140717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.140705    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.140809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.140798    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.141075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.141053    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.141224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.141207    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.141394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.141377    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.141590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.141572    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.142291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.142269    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.143203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.143182    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.144370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.144348    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.167626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.163184    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.168397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.168365    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.180067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.178325    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.181086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.181025    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.238077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.238007    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bsb5\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.244156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.243570    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\") pod \"e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") " pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.244156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.243872    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:34.669186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.669154    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:31:34.669706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.669674    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy" containerID="cri-o://a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15" gracePeriod=30
Nov 01 03:31:34.868001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.866951    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:31:34.953161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.948771    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:31:34.953161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.948888    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:31:34.953161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:34.948930    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:31:35.648821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:35.648778    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerStarted","Data":"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"}
Nov 01 03:31:35.649058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:35.648841    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerStarted","Data":"5bbf6ff48e69f2f28cf488491a87280e8f5ef223c31bd91d22cdb4c97eef88db"}
Nov 01 03:31:35.771551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:35.771495    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podStartSLOduration=2.7714811729999997 podStartE2EDuration="2.771481173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:31:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:31:35.769718929 +0000 UTC m=+1462.185618460" watchObservedRunningTime="2026-11-01 03:31:35.771481173 +0000 UTC m=+1462.187380702"
Nov 01 03:31:36.247203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:36.247151    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:36.271001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:36.269342    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:36.664107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:36.663852    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:36.668880 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:36.668852    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:31:39.855714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.855688    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:31:39.962751 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.962663    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.962914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.962781    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.962914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.962862    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.962914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.962909    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.962974    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963149 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963111    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963149 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963124    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:39.963260 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963223    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963369 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963343    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963521 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963373    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:39.963521 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963416    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\") pod \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\" (UID: \"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c\") "
Nov 01 03:31:39.963652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963561    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:31:39.963716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963658    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:39.963869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.963847    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:39.964291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.964255    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:39.964381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.964294    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:39.964381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.964310    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:39.964381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.964326    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:39.964381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.964340    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:39.965489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.965453    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:31:39.965587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.965549    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:31:39.965677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.965660    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:39.965840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:39.965827    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz" pod "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" (UID: "0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c"). InnerVolumeSpecName "kube-api-access-tl5sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:31:40.066385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.066349    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl5sz\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-kube-api-access-tl5sz\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:40.066385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.066384    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:40.066385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.066397    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:40.066709 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.066406    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:40.701909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.701821    4892 generic.go:356] "Generic (PLEG): container finished" podID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerID="a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15" exitCode=0
Nov 01 03:31:40.701909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.701875    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerDied","Data":"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"}
Nov 01 03:31:40.701909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.701909    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx" event={"ID":"0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c","Type":"ContainerDied","Data":"e0036f93fe104eea9bb43db30d32d56b0584cd3d33aa93f271a54216ea948367"}
Nov 01 03:31:40.702230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.701938    4892 scope.go:122] "RemoveContainer" containerID="a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"
Nov 01 03:31:40.702230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.701873    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"
Nov 01 03:31:40.717516 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.717494    4892 scope.go:122] "RemoveContainer" containerID="c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"
Nov 01 03:31:40.727455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.727434    4892 scope.go:122] "RemoveContainer" containerID="a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"
Nov 01 03:31:40.727768 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:40.727739    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15\": container with ID starting with a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15 not found: ID does not exist" containerID="a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"
Nov 01 03:31:40.727819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.727779    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15"} err="failed to get container status \"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15\": rpc error: code = NotFound desc = could not find container \"a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15\": container with ID starting with a3d05196a2e7d7b529f8b2aec72365b25c3e12d99a4c4d6160f9c4768a415a15 not found: ID does not exist"
Nov 01 03:31:40.727819 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.727799    4892 scope.go:122] "RemoveContainer" containerID="c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"
Nov 01 03:31:40.728089 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:40.728067    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc\": container with ID starting with c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc not found: ID does not exist" containerID="c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"
Nov 01 03:31:40.728164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.728095    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc"} err="failed to get container status \"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc\": rpc error: code = NotFound desc = could not find container \"c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc\": container with ID starting with c0b69fe78051bbb87dbbf21d5552a7d4c862804a6d52ef7a1edaf1ae874e6fcc not found: ID does not exist"
Nov 01 03:31:40.936055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:40.936009    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:31:41.015006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.012590    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7911e5a0-openshift-default-5d9d8d796c-7mslx"]
Nov 01 03:31:41.444110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.442967    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:31:41.444110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.443835    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:31:41.444110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.443854    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:31:41.444110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.444056    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:31:41.448693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.448662    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.501134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.501094    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqzcs\" (UniqueName: \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.501345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.501325    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.502221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.502188    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-d262c155\""
Nov 01 03:31:41.507076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.507051    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-d262c155-dockercfg-dhttk\""
Nov 01 03:31:41.507782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.507759    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:31:41.602854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.602818    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.603119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.603089    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqzcs\" (UniqueName: \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.615894 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.615854    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.665888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.665850    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqzcs\" (UniqueName: \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\") pod \"payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") " pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.776277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.776107    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:41.800266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:41.799475    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" path="/var/lib/kubelet/pods/0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c/volumes"
Nov 01 03:31:42.224991 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:42.224946    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:31:42.226245 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:31:42.226182    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37fda2d8_3ed6_4d85_b7e1_9e569d133b54.slice/crio-6b8a61deff4d45cfe0b88e6fb25c6cf6dc8b6749d158d783cb43f8e7b2db5fd5 WatchSource:0}: Error finding container 6b8a61deff4d45cfe0b88e6fb25c6cf6dc8b6749d158d783cb43f8e7b2db5fd5: Status 404 returned error can't find the container with id 6b8a61deff4d45cfe0b88e6fb25c6cf6dc8b6749d158d783cb43f8e7b2db5fd5
Nov 01 03:31:42.809769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:42.809729    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" event={"ID":"37fda2d8-3ed6-4d85-b7e1-9e569d133b54","Type":"ContainerStarted","Data":"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"}
Nov 01 03:31:42.809967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:42.809800    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:42.809967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:42.809823    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" event={"ID":"37fda2d8-3ed6-4d85-b7e1-9e569d133b54","Type":"ContainerStarted","Data":"6b8a61deff4d45cfe0b88e6fb25c6cf6dc8b6749d158d783cb43f8e7b2db5fd5"}
Nov 01 03:31:42.930157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:42.930105    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" podStartSLOduration=1.930089013 podStartE2EDuration="1.930089013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:31:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:31:42.926492481 +0000 UTC m=+1469.342392023" watchObservedRunningTime="2026-11-01 03:31:42.930089013 +0000 UTC m=+1469.345988570"
Nov 01 03:31:46.971006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:46.962540    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:31:46.971006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:46.962810    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy" containerID="cri-o://7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48" gracePeriod=30
Nov 01 03:31:52.258706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.258672    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:31:52.322734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322705    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.322923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322779    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.322923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322824    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.322923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322888    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.322923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322911    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.323223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322935    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.323223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.322990    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.323223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323097    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.323379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323282    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:52.323379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323310    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:52.323379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323330    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\") pod \"2e865d99-cde7-49f0-824d-c84d3a5d5082\" (UID: \"2e865d99-cde7-49f0-824d-c84d3a5d5082\") "
Nov 01 03:31:52.323379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323336    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:52.323784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323606    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:52.323784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.323629    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:31:52.324066 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.324048    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e865d99-cde7-49f0-824d-c84d3a5d5082-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.324165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.324071    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.324165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.324082    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.324165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.324091    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.324165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.324100    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.325414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.325388    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:31:52.325524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.325483    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:31:52.325574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.325562    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "kube-api-access-sx994". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:31:52.325616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.325601    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo" pod "2e865d99-cde7-49f0-824d-c84d3a5d5082" (UID: "2e865d99-cde7-49f0-824d-c84d3a5d5082"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:31:52.426360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.426326    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.426360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.426357    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx994\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-kube-api-access-sx994\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.426360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.426368    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.426683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.426377    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e865d99-cde7-49f0-824d-c84d3a5d5082-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:31:52.886747 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.886709    4892 generic.go:356] "Generic (PLEG): container finished" podID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerID="7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48" exitCode=0
Nov 01 03:31:52.886933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.886755    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerDied","Data":"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"}
Nov 01 03:31:52.886933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.886761    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"
Nov 01 03:31:52.886933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.886796    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz" event={"ID":"2e865d99-cde7-49f0-824d-c84d3a5d5082","Type":"ContainerDied","Data":"c9d0d6769282b94fd9b0c47f467393eb8ac40ddef0854e71b1c22d1c669ecda4"}
Nov 01 03:31:52.886933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.886818    4892 scope.go:122] "RemoveContainer" containerID="7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"
Nov 01 03:31:52.898220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.898195    4892 scope.go:122] "RemoveContainer" containerID="743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"
Nov 01 03:31:52.907561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.907527    4892 scope.go:122] "RemoveContainer" containerID="7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"
Nov 01 03:31:52.907837 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:52.907818    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48\": container with ID starting with 7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48 not found: ID does not exist" containerID="7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"
Nov 01 03:31:52.907902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.907847    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48"} err="failed to get container status \"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48\": rpc error: code = NotFound desc = could not find container \"7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48\": container with ID starting with 7132597ab994580c9f85d6e9dc5cb981a865cc04694b8fcad2327eae58c37a48 not found: ID does not exist"
Nov 01 03:31:52.907902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.907867    4892 scope.go:122] "RemoveContainer" containerID="743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"
Nov 01 03:31:52.908167 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:52.908150    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801\": container with ID starting with 743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801 not found: ID does not exist" containerID="743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"
Nov 01 03:31:52.908230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:52.908178    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801"} err="failed to get container status \"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801\": rpc error: code = NotFound desc = could not find container \"743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801\": container with ID starting with 743933c4310ee75cb371ba83440b4643b40a97683199e4d8d4a5956a6fa30801 not found: ID does not exist"
Nov 01 03:31:53.055058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:53.055023    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:31:53.105446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:53.105404    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e6e7ea12-openshift-default-7cb5447b8f-sfpnz"]
Nov 01 03:31:53.702893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:53.702859    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" path="/var/lib/kubelet/pods/2e865d99-cde7-49f0-824d-c84d3a5d5082/volumes"
Nov 01 03:31:53.823031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:31:53.822969    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:31:57.614862 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:31:57.614816    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:32:08.360004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:08.323678    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" probeResult="failure" output="Get \"https://10.134.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:32:08.477009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:08.420839    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" probeResult="failure" output="Get \"https://10.134.0.49:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:32:08.477009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:08.454084    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:32:08.545217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:08.544716    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:32:08.657487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:08.657344    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:32:13.864053 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:13.864019    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:32:13.866653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:13.866631    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:32:27.613050 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:32:27.612972    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:32:39.025934 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.025899    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:32:39.026860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026836    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:32:39.027019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026863    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:32:39.027019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026905    4892 cpu_manager.go:414] "containerMap: removing container" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:39.027019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026913    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:39.027019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026955    4892 cpu_manager.go:414] "containerMap: removing container" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:39.027019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.026963    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:39.027719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.027696    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b7df0b3-0528-4e22-b2f0-3f7ecb7f7b1c" containerName="istio-proxy"
Nov 01 03:32:39.027864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.027837    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:39.037192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.037164    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.187502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.187466    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-2058fb-openshift-default-dockercfg-mvxvt\""
Nov 01 03:32:39.193103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193068    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.193419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193397    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.193593 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193576    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l5lt\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.193702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193688    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.193842 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193829    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.193957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.193941    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.194147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.194116    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.194328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.194291    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.194445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.194430    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.303067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.287246    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.306827    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l5lt\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.306881    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.306963    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307029    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307108    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307155    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307189    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307296    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.307437    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.310007 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.308479    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.324134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.323370    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.324134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.323638    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.334892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.330325    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.334892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.332575    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.344916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.344881    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.404484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.387606    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.404484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.400314    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l5lt\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.416229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.411910    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\") pod \"e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") " pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:39.660284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:39.660186    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:40.114259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.114228    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:32:40.115871 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:32:40.115841    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec7bfac_b2a1_4c3d_8726_5f3f2e78ce97.slice/crio-6b4ea4bfda77351b520c7f200c61df13a778a28ff1b8f9834f9bb7b067fdb0a2 WatchSource:0}: Error finding container 6b4ea4bfda77351b520c7f200c61df13a778a28ff1b8f9834f9bb7b067fdb0a2: Status 404 returned error can't find the container with id 6b4ea4bfda77351b520c7f200c61df13a778a28ff1b8f9834f9bb7b067fdb0a2
Nov 01 03:32:40.118899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.118857    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:40.119048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.118951    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:40.119048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.119035    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:40.455005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.445538    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerStarted","Data":"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"}
Nov 01 03:32:40.455005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.445612    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerStarted","Data":"6b4ea4bfda77351b520c7f200c61df13a778a28ff1b8f9834f9bb7b067fdb0a2"}
Nov 01 03:32:40.587194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.587128    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podStartSLOduration=2.587102247 podStartE2EDuration="2.587102247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:32:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:32:40.572611453 +0000 UTC m=+1526.988510992" watchObservedRunningTime="2026-11-01 03:32:40.587102247 +0000 UTC m=+1527.003001777"
Nov 01 03:32:40.660763 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:40.660667    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:41.667577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:41.667538    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:42.461968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:42.461934    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:42.463091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:42.463068    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:32:47.068297 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.068206    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:32:47.069939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.069910    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e865d99-cde7-49f0-824d-c84d3a5d5082" containerName="istio-proxy"
Nov 01 03:32:47.074640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.074605    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.128463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.128421    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77l5b\" (UniqueName: \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.128683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.128660    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.140739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.140702    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w4-2058fb\""
Nov 01 03:32:47.204044 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.190064    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w4-2058fb-dockercfg-l7c85\""
Nov 01 03:32:47.204044 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.196783    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:32:47.228290 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.228251    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:32:47.232854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.232817    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77l5b\" (UniqueName: \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.233126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.233039    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.234101 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.234063    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.243814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.243780    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.334405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.334278    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.334745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.334719    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79mwh\" (UniqueName: \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.341964 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.340662    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:32:47.370073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.353800    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77l5b\" (UniqueName: \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\") pod \"payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.418860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.409309    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:47.452085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.451842    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.452085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.452048    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79mwh\" (UniqueName: \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.521006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.518394    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.525567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.522203    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79mwh\" (UniqueName: \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\") pod \"payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.563469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.563391    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:47.949327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:47.949252    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:32:47.990332 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:32:47.990295    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14627dce_7a61_4891_ab62_47b1082b875b.slice/crio-5f5d22fc950d26dc347ff8058070f871f555d6f8e1b5358e9d679e16230bf437 WatchSource:0}: Error finding container 5f5d22fc950d26dc347ff8058070f871f555d6f8e1b5358e9d679e16230bf437: Status 404 returned error can't find the container with id 5f5d22fc950d26dc347ff8058070f871f555d6f8e1b5358e9d679e16230bf437
Nov 01 03:32:48.001046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.001019    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:32:48.004241 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:32:48.004209    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20623345_4e6b_4b43_9c95_a98412d8cf91.slice/crio-e86ff668b12329f6134fdfa989751f918ef361c5ef3694d1811ea519e9bb98bc WatchSource:0}: Error finding container e86ff668b12329f6134fdfa989751f918ef361c5ef3694d1811ea519e9bb98bc: Status 404 returned error can't find the container with id e86ff668b12329f6134fdfa989751f918ef361c5ef3694d1811ea519e9bb98bc
Nov 01 03:32:48.557824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.557784    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" event={"ID":"20623345-4e6b-4b43-9c95-a98412d8cf91","Type":"ContainerStarted","Data":"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"}
Nov 01 03:32:48.558342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.557859    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:32:48.558342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.557885    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" event={"ID":"20623345-4e6b-4b43-9c95-a98412d8cf91","Type":"ContainerStarted","Data":"e86ff668b12329f6134fdfa989751f918ef361c5ef3694d1811ea519e9bb98bc"}
Nov 01 03:32:48.559716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.559688    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" event={"ID":"14627dce-7a61-4891-ab62-47b1082b875b","Type":"ContainerStarted","Data":"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"}
Nov 01 03:32:48.559850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.559737    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" event={"ID":"14627dce-7a61-4891-ab62-47b1082b875b","Type":"ContainerStarted","Data":"5f5d22fc950d26dc347ff8058070f871f555d6f8e1b5358e9d679e16230bf437"}
Nov 01 03:32:48.559850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.559771    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:48.676374 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.676318    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" podStartSLOduration=1.6763033059999999 podStartE2EDuration="1.676303306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:32:48.675096576 +0000 UTC m=+1535.090996106" watchObservedRunningTime="2026-11-01 03:32:48.676303306 +0000 UTC m=+1535.092202833"
Nov 01 03:32:48.818743 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:48.798780    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" podStartSLOduration=1.798756064 podStartE2EDuration="1.798756064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:32:48.791387544 +0000 UTC m=+1535.207287074" watchObservedRunningTime="2026-11-01 03:32:48.798756064 +0000 UTC m=+1535.214655593"
Nov 01 03:32:51.582540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.582501    4892 generic.go:356] "Generic (PLEG): container finished" podID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerID="b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf" exitCode=0
Nov 01 03:32:51.582995 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.582575    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerDied","Data":"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"}
Nov 01 03:32:51.583085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.583068    4892 scope.go:122] "RemoveContainer" containerID="b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"
Nov 01 03:32:51.584777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.584748    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:51.584840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.584816    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:51.584877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.584855    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:32:51.663815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:51.663789    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:32:52.592045 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:52.592003    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerStarted","Data":"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"}
Nov 01 03:32:53.246005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:53.245354    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:32:53.247287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:53.247259    4892 patch_prober.go:29] interesting pod/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Nov 01 03:32:53.247418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:53.247310    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Nov 01 03:32:54.253006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:54.245399    4892 patch_prober.go:29] interesting pod/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Nov 01 03:32:54.253006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:54.245486    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Nov 01 03:32:55.246473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:55.246432    4892 patch_prober.go:29] interesting pod/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Nov 01 03:32:55.246662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:55.246502    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Nov 01 03:32:56.246532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:56.246496    4892 patch_prober.go:29] interesting pod/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Nov 01 03:32:56.247051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:56.247021    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Nov 01 03:32:57.246518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:57.246471    4892 patch_prober.go:29] interesting pod/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused" start-of-body=
Nov 01 03:32:57.246717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:57.246561    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.84:15021/healthz/ready\": dial tcp 10.134.0.84:15021: connect: connection refused"
Nov 01 03:32:57.614967 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:32:57.614929    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:32:58.250079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:58.250046    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:32:58.250493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:58.250472    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:32:58.251199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:58.251176    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:32:59.570067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:59.570035    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:32:59.570693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:32:59.570127    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:33:20.050736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:20.050697    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"]
Nov 01 03:33:20.100626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:20.100590    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6e66f2ea-dlnqt"]
Nov 01 03:33:21.703824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:21.703789    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e37e1014-0507-4902-a61e-d3f86f0796eb" path="/var/lib/kubelet/pods/e37e1014-0507-4902-a61e-d3f86f0796eb/volumes"
Nov 01 03:33:27.613546 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:33:27.613505    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:33:37.222428 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:37.222391    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:33:42.223488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.223425    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" podUID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" containerName="payload-processing" containerID="cri-o://896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e" gracePeriod=25
Nov 01 03:33:42.546473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.546444    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:33:42.621479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.621441    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\" (UniqueName: \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\") pod \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") "
Nov 01 03:33:42.621683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.621543    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\") pod \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\" (UID: \"37fda2d8-3ed6-4d85-b7e1-9e569d133b54\") "
Nov 01 03:33:42.621962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.621938    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume" pod "37fda2d8-3ed6-4d85-b7e1-9e569d133b54" (UID: "37fda2d8-3ed6-4d85-b7e1-9e569d133b54"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:33:42.622322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.622303    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:33:42.623802 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.623778    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs" pod "37fda2d8-3ed6-4d85-b7e1-9e569d133b54" (UID: "37fda2d8-3ed6-4d85-b7e1-9e569d133b54"). InnerVolumeSpecName "kube-api-access-dqzcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:33:42.724237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:42.724193    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqzcs\" (UniqueName: \"kubernetes.io/projected/37fda2d8-3ed6-4d85-b7e1-9e569d133b54-kube-api-access-dqzcs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:33:43.102846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.102807    4892 generic.go:356] "Generic (PLEG): container finished" podID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" containerID="896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e" exitCode=0
Nov 01 03:33:43.103040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.102862    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"
Nov 01 03:33:43.103040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.102891    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" event={"ID":"37fda2d8-3ed6-4d85-b7e1-9e569d133b54","Type":"ContainerDied","Data":"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"}
Nov 01 03:33:43.103040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.102948    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g" event={"ID":"37fda2d8-3ed6-4d85-b7e1-9e569d133b54","Type":"ContainerDied","Data":"6b8a61deff4d45cfe0b88e6fb25c6cf6dc8b6749d158d783cb43f8e7b2db5fd5"}
Nov 01 03:33:43.103040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.102970    4892 scope.go:122] "RemoveContainer" containerID="896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"
Nov 01 03:33:43.115653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.115626    4892 scope.go:122] "RemoveContainer" containerID="896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"
Nov 01 03:33:43.116014 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:33:43.115988    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e\": container with ID starting with 896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e not found: ID does not exist" containerID="896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"
Nov 01 03:33:43.116093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.116026    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e"} err="failed to get container status \"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e\": rpc error: code = NotFound desc = could not find container \"896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e\": container with ID starting with 896f00f0b5e4429967c80d18c14e15d8512342effebbd0e74b277d4519453e9e not found: ID does not exist"
Nov 01 03:33:43.269973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.269935    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:33:43.277928 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.277891    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-d262c155-7f799685b9-rq65g"]
Nov 01 03:33:43.702858 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:43.702827    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" path="/var/lib/kubelet/pods/37fda2d8-3ed6-4d85-b7e1-9e569d133b54/volumes"
Nov 01 03:33:56.938833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.938783    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:33:56.940614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.940587    4892 cpu_manager.go:414] "containerMap: removing container" podUID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" containerName="payload-processing"
Nov 01 03:33:56.940792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.940776    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" containerName="payload-processing"
Nov 01 03:33:56.941088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.941070    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="37fda2d8-3ed6-4d85-b7e1-9e569d133b54" containerName="payload-processing"
Nov 01 03:33:56.969937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.969900    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:56.986422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:56.986392    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:33:57.031782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.031734    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5456n\" (UniqueName: \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.032124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.032099    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.093200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.093150    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-fdbdfefd\""
Nov 01 03:33:57.135990 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.133847    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5456n\" (UniqueName: \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.135990 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.134144    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.135990 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.135324    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.142411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.142386    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-fdbdfefd-dockercfg-lxkv8\""
Nov 01 03:33:57.200946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.200850    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5456n\" (UniqueName: \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\") pod \"payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") " pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.295174 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.295141    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:57.614188 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:33:57.614148    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:33:57.711241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:57.711208    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:33:57.711882 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:33:57.711851    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod175a95b4_59f0_4d78_9c11_10abc8ef3c2b.slice/crio-96fd83fe7ed15f7822d75983f316e83c80ddd4e81b5c29ba86891ebc05377ae3 WatchSource:0}: Error finding container 96fd83fe7ed15f7822d75983f316e83c80ddd4e81b5c29ba86891ebc05377ae3: Status 404 returned error can't find the container with id 96fd83fe7ed15f7822d75983f316e83c80ddd4e81b5c29ba86891ebc05377ae3
Nov 01 03:33:58.232943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:58.232905    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" event={"ID":"175a95b4-59f0-4d78-9c11-10abc8ef3c2b","Type":"ContainerStarted","Data":"75100b94b63229f56653a0069c34d7f8c376c3a8d6707fdec1487f046e33a1ab"}
Nov 01 03:33:58.233409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:58.232969    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" event={"ID":"175a95b4-59f0-4d78-9c11-10abc8ef3c2b","Type":"ContainerStarted","Data":"96fd83fe7ed15f7822d75983f316e83c80ddd4e81b5c29ba86891ebc05377ae3"}
Nov 01 03:33:58.233409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:58.233032    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:33:58.352623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:33:58.352562    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" podStartSLOduration=2.352546328 podStartE2EDuration="2.352546328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:33:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:33:58.350434616 +0000 UTC m=+1604.766334145" watchObservedRunningTime="2026-11-01 03:33:58.352546328 +0000 UTC m=+1604.768445858"
Nov 01 03:34:09.244659 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:09.244627    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:34:17.918481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:17.918447    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:34:17.919583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:17.919534    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy" containerID="cri-o://b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623" gracePeriod=30
Nov 01 03:34:23.193740 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.193708    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:34:23.260525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260484    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.260697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260559    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.260697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260590    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.260697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260640    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.260891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260722    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.260891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260812    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:23.260891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260854    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.261072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260897    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.261072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.260927    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:23.261183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261065    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:23.261183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261114    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.261287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261180    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\") pod \"e65a9af5-11d0-4d5f-8fc3-b54293930366\" (UID: \"e65a9af5-11d0-4d5f-8fc3-b54293930366\") "
Nov 01 03:34:23.261410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261376    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:23.261669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261641    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:34:23.261937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261917    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.262026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261945    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e65a9af5-11d0-4d5f-8fc3-b54293930366-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.262026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261959    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.262026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.261973    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.262136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.262025    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.263379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.263344    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "kube-api-access-7bsb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:23.263943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.263851    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:23.263943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.263924    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:34:23.264127 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.263974    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token" pod "e65a9af5-11d0-4d5f-8fc3-b54293930366" (UID: "e65a9af5-11d0-4d5f-8fc3-b54293930366"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:23.363595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.363552    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.363595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.363583    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.363595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.363593    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bsb5\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-kube-api-access-7bsb5\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.363595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.363603    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e65a9af5-11d0-4d5f-8fc3-b54293930366-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:23.454904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.454857    4892 generic.go:356] "Generic (PLEG): container finished" podID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerID="b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623" exitCode=0
Nov 01 03:34:23.455122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.454970    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerDied","Data":"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"}
Nov 01 03:34:23.455122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.455068    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8" event={"ID":"e65a9af5-11d0-4d5f-8fc3-b54293930366","Type":"ContainerDied","Data":"5bbf6ff48e69f2f28cf488491a87280e8f5ef223c31bd91d22cdb4c97eef88db"}
Nov 01 03:34:23.455122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.455097    4892 scope.go:122] "RemoveContainer" containerID="b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"
Nov 01 03:34:23.455302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.455258    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"
Nov 01 03:34:23.473478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.473458    4892 scope.go:122] "RemoveContainer" containerID="b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"
Nov 01 03:34:23.484578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.484549    4892 scope.go:122] "RemoveContainer" containerID="b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"
Nov 01 03:34:23.484902 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:34:23.484877    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623\": container with ID starting with b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623 not found: ID does not exist" containerID="b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"
Nov 01 03:34:23.485011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.484913    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623"} err="failed to get container status \"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623\": rpc error: code = NotFound desc = could not find container \"b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623\": container with ID starting with b1a918485c01d1df1a0a98a74db77506924679cc609914ce56196e997feaa623 not found: ID does not exist"
Nov 01 03:34:23.485011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.484934    4892 scope.go:122] "RemoveContainer" containerID="b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"
Nov 01 03:34:23.485237 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:34:23.485217    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf\": container with ID starting with b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf not found: ID does not exist" containerID="b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"
Nov 01 03:34:23.485300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.485243    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf"} err="failed to get container status \"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf\": rpc error: code = NotFound desc = could not find container \"b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf\": container with ID starting with b6374273130d8e89193f6004ad7e387d3530e418fb950536f563aa84b0e7dbaf not found: ID does not exist"
Nov 01 03:34:23.631171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.631086    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:34:23.636934 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.636902    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d262c155-gw-openshift-default-c45f8b8b9-c6vl8"]
Nov 01 03:34:23.702973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:23.702944    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" path="/var/lib/kubelet/pods/e65a9af5-11d0-4d5f-8fc3-b54293930366/volumes"
Nov 01 03:34:27.885013 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:34:27.865372    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:34:30.184950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.184900    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:34:30.185900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.185843    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" containerID="cri-o://f1ab6ca8d782099ab1e4f5a4dc77c0d9295265ff2c832c4b6f22b29a4923933c" gracePeriod=30
Nov 01 03:34:30.213056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.213017    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:34:30.213346 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.213301    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy" containerID="cri-o://21e562952fb8d7bb0dc18716b0289c4053e9186e999ff7a64fcd040da393cd8e" gracePeriod=30
Nov 01 03:34:30.217216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.217178    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:34:30.217468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.217423    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy" containerID="cri-o://462397bdc0070e5e4ea8d8e4a4e6bbea95532b367bb82f262fc93017a826192f" gracePeriod=30
Nov 01 03:34:30.262706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.262660    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:34:30.262955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:30.262913    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy" containerID="cri-o://8a4713369a7468c0511c5b2f04a2d1fbabd826241594ade31e5277018ee31d11" gracePeriod=30
Nov 01 03:34:34.103324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.103282    4892 patch_prober.go:29] interesting pod/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:34:34.103729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.103362    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:34:34.565429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.565386    4892 generic.go:356] "Generic (PLEG): container finished" podID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerID="0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83" exitCode=0
Nov 01 03:34:34.565713 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.565517    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerDied","Data":"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"}
Nov 01 03:34:34.566193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.566171    4892 scope.go:122] "RemoveContainer" containerID="0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"
Nov 01 03:34:34.568061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.568001    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:34:34.568190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.568088    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:34:34.568190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:34.568122    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:34:35.581364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.581191    4892 generic.go:356] "Generic (PLEG): container finished" podID="2e16e809-93d0-4fa3-86c0-770c32921326" containerID="21e562952fb8d7bb0dc18716b0289c4053e9186e999ff7a64fcd040da393cd8e" exitCode=0
Nov 01 03:34:35.581364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.581272    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" event={"ID":"2e16e809-93d0-4fa3-86c0-770c32921326","Type":"ContainerDied","Data":"21e562952fb8d7bb0dc18716b0289c4053e9186e999ff7a64fcd040da393cd8e"}
Nov 01 03:34:35.585435 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.585404    4892 generic.go:356] "Generic (PLEG): container finished" podID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerID="462397bdc0070e5e4ea8d8e4a4e6bbea95532b367bb82f262fc93017a826192f" exitCode=0
Nov 01 03:34:35.585561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.585517    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerDied","Data":"462397bdc0070e5e4ea8d8e4a4e6bbea95532b367bb82f262fc93017a826192f"}
Nov 01 03:34:35.585623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.585573    4892 scope.go:122] "RemoveContainer" containerID="6117ddfb846785ed33005921b28fe8656602ad531717150bccb65130d15278df"
Nov 01 03:34:35.589382 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.589360    4892 generic.go:356] "Generic (PLEG): container finished" podID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerID="8a4713369a7468c0511c5b2f04a2d1fbabd826241594ade31e5277018ee31d11" exitCode=0
Nov 01 03:34:35.589741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.589719    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" event={"ID":"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0","Type":"ContainerDied","Data":"8a4713369a7468c0511c5b2f04a2d1fbabd826241594ade31e5277018ee31d11"}
Nov 01 03:34:35.592162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.592137    4892 generic.go:356] "Generic (PLEG): container finished" podID="d6ade366-fa02-4a60-b751-9e652e53f260" containerID="f1ab6ca8d782099ab1e4f5a4dc77c0d9295265ff2c832c4b6f22b29a4923933c" exitCode=0
Nov 01 03:34:35.592274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.592248    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" event={"ID":"d6ade366-fa02-4a60-b751-9e652e53f260","Type":"ContainerDied","Data":"f1ab6ca8d782099ab1e4f5a4dc77c0d9295265ff2c832c4b6f22b29a4923933c"}
Nov 01 03:34:35.595100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.595072    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerStarted","Data":"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"}
Nov 01 03:34:35.679184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.668436    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:34:35.682362 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.679732    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:35.682362 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.679787    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:35.682949 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.682912    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:34:35.815669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.815638    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.815832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.815740    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.815832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.815791    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.815933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.815853    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.815953    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816309    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816359    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816467 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816428    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816507    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\") pod \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\" (UID: \"f6e570a1-365e-44f2-b04d-c8f1a3ab898c\") "
Nov 01 03:34:35.816595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816536    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:35.816800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.816778    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:35.818512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.817578    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.818512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.817623    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.820352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.819437    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:34:35.820352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.820201    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:35.820352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.820288    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:35.820894 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.820862    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:34:35.843870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.842217    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:35.844441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.844389    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:35.872626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.871923    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82" pod "f6e570a1-365e-44f2-b04d-c8f1a3ab898c" (UID: "f6e570a1-365e-44f2-b04d-c8f1a3ab898c"). InnerVolumeSpecName "kube-api-access-lbs82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:35.875328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.872900    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:34:35.887231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.887205    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:34:35.918752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.918710    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919288    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbs82\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-kube-api-access-lbs82\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919318    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919333    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919345    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919359    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919372    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:35.919406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:35.919383    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e570a1-365e-44f2-b04d-c8f1a3ab898c-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.020234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020196    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.020455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020254    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.020455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020317    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.020455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020349    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.020455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020386    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.020455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020424    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020488    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020528    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020559    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020613    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020657    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020683    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020710    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.020727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020707    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020739    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020751    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020813    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020852    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020877    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020915    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020949    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.020957    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021006    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021057    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021122    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021164    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.021187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021188    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.059586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.021217    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.059586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.046091    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.059586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.058734    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.060336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.060292    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062156    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "kube-api-access-n8mk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062438    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062618    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.058004    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\") pod \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\" (UID: \"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0\") "
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062700    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062805    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\") pod \"d6ade366-fa02-4a60-b751-9e652e53f260\" (UID: \"d6ade366-fa02-4a60-b751-9e652e53f260\") "
Nov 01 03:34:36.063511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.062837    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\") pod \"2e16e809-93d0-4fa3-86c0-770c32921326\" (UID: \"2e16e809-93d0-4fa3-86c0-770c32921326\") "
Nov 01 03:34:36.064087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.063525    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.064087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.063846    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.064087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.063848    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:34:36.064087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064075    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.064304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064104    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "kube-api-access-k7g4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.064304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064156    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.064454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064317    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:34:36.064454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064311    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.064454 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064371    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.064972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064927    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e16e809-93d0-4fa3-86c0-770c32921326-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.064972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064956    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.064972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.064969    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065002    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065014    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065026    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065039    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065052    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8mk6\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-kube-api-access-n8mk6\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065064    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6ade366-fa02-4a60-b751-9e652e53f260-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065078    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065091    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065104    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.065676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065364    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.065676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.065480    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:34:36.069490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.069456    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.069621 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.069538    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo" pod "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" (UID: "0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:34:36.069688 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.069662    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.070262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.070212    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy" pod "2e16e809-93d0-4fa3-86c0-770c32921326" (UID: "2e16e809-93d0-4fa3-86c0-770c32921326"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:34:36.070262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.070237    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56" pod "d6ade366-fa02-4a60-b751-9e652e53f260" (UID: "d6ade366-fa02-4a60-b751-9e652e53f260"). InnerVolumeSpecName "kube-api-access-8kp56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166460    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166491    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166501    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166509    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166522    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6ade366-fa02-4a60-b751-9e652e53f260-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166532    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kp56\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-kube-api-access-8kp56\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166542    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e16e809-93d0-4fa3-86c0-770c32921326-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166550    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e16e809-93d0-4fa3-86c0-770c32921326-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166558    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6ade366-fa02-4a60-b751-9e652e53f260-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166566    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166575    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7g4c\" (UniqueName: \"kubernetes.io/projected/2e16e809-93d0-4fa3-86c0-770c32921326-kube-api-access-k7g4c\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166583    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6ade366-fa02-4a60-b751-9e652e53f260-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166592    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166602    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.166647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.166611    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:34:36.603263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.603236    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"
Nov 01 03:34:36.603734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.603227    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p" event={"ID":"2e16e809-93d0-4fa3-86c0-770c32921326","Type":"ContainerDied","Data":"d35277d0cf1e5c1186150c3ec9d61f1ac572aa1690947e99e61e3b7091411d7f"}
Nov 01 03:34:36.603734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.603377    4892 scope.go:122] "RemoveContainer" containerID="21e562952fb8d7bb0dc18716b0289c4053e9186e999ff7a64fcd040da393cd8e"
Nov 01 03:34:36.605115 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.605093    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"
Nov 01 03:34:36.605259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.605107    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76" event={"ID":"f6e570a1-365e-44f2-b04d-c8f1a3ab898c","Type":"ContainerDied","Data":"f5b1ec1ac311c563723348f70172f2d7e15eec61746c7f14c8d6f01083455608"}
Nov 01 03:34:36.613668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.613644    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"
Nov 01 03:34:36.613941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.613915    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2" event={"ID":"0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0","Type":"ContainerDied","Data":"53b7255661f2921887bf7b1b6752564b1429b5a8ad51b6a3967e388e84d1d8a7"}
Nov 01 03:34:36.615605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.615577    4892 scope.go:122] "RemoveContainer" containerID="462397bdc0070e5e4ea8d8e4a4e6bbea95532b367bb82f262fc93017a826192f"
Nov 01 03:34:36.643008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.616796    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"
Nov 01 03:34:36.643008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.616809    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq" event={"ID":"d6ade366-fa02-4a60-b751-9e652e53f260","Type":"ContainerDied","Data":"b9a63bd9f597b26dcba6f4392e9fccfbe2bbce18d6652e8963221ef19ff1f5de"}
Nov 01 03:34:36.657163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.657141    4892 scope.go:122] "RemoveContainer" containerID="8a4713369a7468c0511c5b2f04a2d1fbabd826241594ade31e5277018ee31d11"
Nov 01 03:34:36.660778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.660747    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:36.660884 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.660802    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:36.671328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.671290    4892 scope.go:122] "RemoveContainer" containerID="f1ab6ca8d782099ab1e4f5a4dc77c0d9295265ff2c832c4b6f22b29a4923933c"
Nov 01 03:34:36.776252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.776206    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:34:36.787168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.787135    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-2s86p"]
Nov 01 03:34:36.949097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.948955    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:34:36.952573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:36.952542    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-tcfq2"]
Nov 01 03:34:37.149698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.149622    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:34:37.153061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.153035    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-6tw76"]
Nov 01 03:34:37.257581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.257552    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:34:37.266779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.266744    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d7f535-openshift-default-856bb56cf-g85rq"]
Nov 01 03:34:37.660847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.660759    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:37.660847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.660824    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:37.703187 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.703152    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" path="/var/lib/kubelet/pods/0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0/volumes"
Nov 01 03:34:37.703741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.703725    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" path="/var/lib/kubelet/pods/2e16e809-93d0-4fa3-86c0-770c32921326/volumes"
Nov 01 03:34:37.704217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.704202    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" path="/var/lib/kubelet/pods/d6ade366-fa02-4a60-b751-9e652e53f260/volumes"
Nov 01 03:34:37.705031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:37.705016    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" path="/var/lib/kubelet/pods/f6e570a1-365e-44f2-b04d-c8f1a3ab898c/volumes"
Nov 01 03:34:38.660857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:38.660827    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:38.661305 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:38.660878    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:39.661264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:39.661226    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:39.661721 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:39.661292    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:40.660814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:40.660731    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Nov 01 03:34:40.660814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:40.660800    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Nov 01 03:34:41.665704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:41.665665    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:34:41.666138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:41.666085    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:34:41.666877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:41.666852    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:34:52.450240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:52.450178    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" probeResult="failure" output="Get \"https://10.134.0.43:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:34:52.606003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:34:52.605478    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": context deadline exceeded"
Nov 01 03:34:57.615701 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:34:57.615660    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:35:13.977326 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:13.977295    4892 scope.go:122] "RemoveContainer" containerID="f3e53660ce95cd4c043a9754167de87ef2eea35709c3da229c7c80e8bcd264c1"
Nov 01 03:35:15.792541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.792504    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:35:15.792974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.792724    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy" containerID="cri-o://c38397b455da33afaeef81923c7cfbcb768260bd26f355d15e1d28fbeea01246" gracePeriod=30
Nov 01 03:35:15.822449 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.822409    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:35:15.822657 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.822621    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy" containerID="cri-o://f647feda9d1a75ab8523fc5bb39bca77c23ff46ae9f4040202f849a206fc869e" gracePeriod=30
Nov 01 03:35:15.842235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.842177    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:35:15.842440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:15.842406    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy" containerID="cri-o://0427b9f32dd3a3946a7297044dfc66d4e3cc84c0247cc7e07108099289484f44" gracePeriod=30
Nov 01 03:35:16.582717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.582676    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:35:16.582950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.582923    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy" containerID="cri-o://2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c" gracePeriod=30
Nov 01 03:35:16.586631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.586600    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:35:16.586837 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.586815    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy" containerID="cri-o://299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17" gracePeriod=30
Nov 01 03:35:16.632097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.632040    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:35:16.632310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.632277    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy" containerID="cri-o://a9144291f5b13cacc7631e69b36c8588151d53623e315e2fef4fbdad375e889d" gracePeriod=30
Nov 01 03:35:16.636345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.635904    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:35:16.636345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:16.636141    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" containerID="cri-o://a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94" gracePeriod=30
Nov 01 03:35:19.878547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:19.878511    4892 patch_prober.go:29] interesting pod/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:35:19.879053 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:19.878569    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:35:21.035361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.034223    4892 generic.go:356] "Generic (PLEG): container finished" podID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerID="0427b9f32dd3a3946a7297044dfc66d4e3cc84c0247cc7e07108099289484f44" exitCode=0
Nov 01 03:35:21.035361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.034286    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" event={"ID":"5c06d5e6-550f-4454-a6a9-6f5ec557c253","Type":"ContainerDied","Data":"0427b9f32dd3a3946a7297044dfc66d4e3cc84c0247cc7e07108099289484f44"}
Nov 01 03:35:21.040028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.035932    4892 generic.go:356] "Generic (PLEG): container finished" podID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerID="c38397b455da33afaeef81923c7cfbcb768260bd26f355d15e1d28fbeea01246" exitCode=0
Nov 01 03:35:21.040028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.036017    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" event={"ID":"255a94a8-506a-4c39-b9c3-82299a68e6d6","Type":"ContainerDied","Data":"c38397b455da33afaeef81923c7cfbcb768260bd26f355d15e1d28fbeea01246"}
Nov 01 03:35:21.040028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.038848    4892 generic.go:356] "Generic (PLEG): container finished" podID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerID="f647feda9d1a75ab8523fc5bb39bca77c23ff46ae9f4040202f849a206fc869e" exitCode=0
Nov 01 03:35:21.040028 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.039158    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" event={"ID":"678ba707-c2e0-4a8a-bcb3-33390a9441bd","Type":"ContainerDied","Data":"f647feda9d1a75ab8523fc5bb39bca77c23ff46ae9f4040202f849a206fc869e"}
Nov 01 03:35:21.128603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.128571    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:35:21.180559 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.180504    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:35:21.221486 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.221459    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:35:21.267548 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267511    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.267731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267616    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.267731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267697    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.267887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267864    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.267951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267941    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267859    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.268026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.267951    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.268138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268080    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268118    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268146    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268179    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268236    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268265    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268298    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268371    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268413    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268475    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268499    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268523    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\") pod \"255a94a8-506a-4c39-b9c3-82299a68e6d6\" (UID: \"255a94a8-506a-4c39-b9c3-82299a68e6d6\") "
Nov 01 03:35:21.268569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268554    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\") pod \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\" (UID: \"678ba707-c2e0-4a8a-bcb3-33390a9441bd\") "
Nov 01 03:35:21.268824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268595    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.268824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268473    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:21.268824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268801    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.269099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.268882    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.269268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269241    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.269613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269430    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.269613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269582    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:21.269891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269845    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.269891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269861    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.269891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269882    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269896    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269909    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269922    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269935    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269949    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269962    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/255a94a8-506a-4c39-b9c3-82299a68e6d6-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.269993    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.270668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.270642    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.270794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.270707    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:21.270969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.270944    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "kube-api-access-p7drz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.271218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.271193    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.271313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.271223    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy" pod "255a94a8-506a-4c39-b9c3-82299a68e6d6" (UID: "255a94a8-506a-4c39-b9c3-82299a68e6d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.271628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.271606    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:21.271831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.271816    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "kube-api-access-kwqb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.272346 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.272328    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token" pod "678ba707-c2e0-4a8a-bcb3-33390a9441bd" (UID: "678ba707-c2e0-4a8a-bcb3-33390a9441bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.371310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371158    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371296    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371321    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371348    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371400    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371397    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.371561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371507    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371646    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371659    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371699    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371733    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371759    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:21.371825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371814    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\") pod \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\" (UID: \"5c06d5e6-550f-4454-a6a9-6f5ec557c253\") "
Nov 01 03:35:21.372191 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.371937    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372583    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372607    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372622    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372635    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/255a94a8-506a-4c39-b9c3-82299a68e6d6-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372648    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372662    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372677    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwqb2\" (UniqueName: \"kubernetes.io/projected/678ba707-c2e0-4a8a-bcb3-33390a9441bd-kube-api-access-kwqb2\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372692    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372707    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7drz\" (UniqueName: \"kubernetes.io/projected/255a94a8-506a-4c39-b9c3-82299a68e6d6-kube-api-access-p7drz\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372723    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372737    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/678ba707-c2e0-4a8a-bcb3-33390a9441bd-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372751    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372764    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.372780    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.374183    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "kube-api-access-zhxll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.374290    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.374377    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:21.375266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.374444    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy" pod "5c06d5e6-550f-4454-a6a9-6f5ec557c253" (UID: "5c06d5e6-550f-4454-a6a9-6f5ec557c253"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:21.474808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.474767    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhxll\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-kube-api-access-zhxll\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.474808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.474810    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.475116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.474828    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.475116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.474843    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c06d5e6-550f-4454-a6a9-6f5ec557c253-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:21.583678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.583639    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:35:21.584608 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584434    4892 cpu_manager.go:414] "containerMap: removing container" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy"
Nov 01 03:35:21.584858 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584835    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584871    4892 cpu_manager.go:414] "containerMap: removing container" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584880    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584890    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584898    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584925    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584933    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584942    4892 cpu_manager.go:414] "containerMap: removing container" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy"
Nov 01 03:35:21.584957 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584949    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.584993    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585001    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585026    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585033    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585048    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585056    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585087    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585094    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585113    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585120    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585284    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e16e809-93d0-4fa3-86c0-770c32921326" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585297    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585306    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585361    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c7a2a0f-1a45-448e-b0b9-3c8b2d26e4f0" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585372    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585388    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585410    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e65a9af5-11d0-4d5f-8fc3-b54293930366" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585420    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585432    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6e570a1-365e-44f2-b04d-c8f1a3ab898c" containerName="istio-proxy"
Nov 01 03:35:21.585437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.585441    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d6ade366-fa02-4a60-b751-9e652e53f260" containerName="istio-proxy"
Nov 01 03:35:21.592490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.592468    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.648219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.648186    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:35:21.649097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.649064    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-0cb10b-serving-cert\""
Nov 01 03:35:21.678749 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.678721    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.678938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.678917    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.679013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.678956    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.679355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.679328    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9llsd\" (UniqueName: \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.780210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.780177    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.780373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.780220    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.781603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.780479    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9llsd\" (UniqueName: \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.781603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.780517    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.781603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.781194    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.781603 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:21.781496    4892 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-serving-cert: secret "maas-api-e2e-shared-a-w1-0cb10b-serving-cert" not found
Nov 01 03:35:21.781603 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:21.781572    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls podName:dbb72c63-aa0d-4bc1-977f-96272a4560b3 nodeName:}" failed. No retries permitted until 2026-11-01 03:35:22.281553107 +0000 UTC m=+1688.697452617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls") pod "maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" (UID: "dbb72c63-aa0d-4bc1-977f-96272a4560b3") : secret "maas-api-e2e-shared-a-w1-0cb10b-serving-cert" not found
Nov 01 03:35:21.795582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.795534    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.851512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.851451    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9llsd\" (UniqueName: \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:21.965766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:21.965736    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:35:22.029865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.029831    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:35:22.035088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.035009    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:35:22.051609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.051576    4892 generic.go:356] "Generic (PLEG): container finished" podID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerID="299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17" exitCode=0
Nov 01 03:35:22.052123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.051713    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" event={"ID":"3ea25d44-8cda-466d-9299-50d80ce9f04f","Type":"ContainerDied","Data":"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"}
Nov 01 03:35:22.052123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.051755    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l" event={"ID":"3ea25d44-8cda-466d-9299-50d80ce9f04f","Type":"ContainerDied","Data":"2153f3384b433fe7b5fd383157ca5bb806cf20d576a0364821f64e5d41ee11b4"}
Nov 01 03:35:22.052123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.051784    4892 scope.go:122] "RemoveContainer" containerID="299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"
Nov 01 03:35:22.052123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.051901    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"
Nov 01 03:35:22.054306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.054276    4892 generic.go:356] "Generic (PLEG): container finished" podID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerID="a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94" exitCode=0
Nov 01 03:35:22.054394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.054375    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"
Nov 01 03:35:22.054547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.054373    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" event={"ID":"ca4e8f78-70a6-449d-9dff-86629bb521b4","Type":"ContainerDied","Data":"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"}
Nov 01 03:35:22.054629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.054588    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4" event={"ID":"ca4e8f78-70a6-449d-9dff-86629bb521b4","Type":"ContainerDied","Data":"3a3da9194f64ee1426f9abae23dfac5695735b9863dab3609f0ece11fadbbede"}
Nov 01 03:35:22.056153 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.056080    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w" event={"ID":"5c06d5e6-550f-4454-a6a9-6f5ec557c253","Type":"ContainerDied","Data":"f81dfc4fadc08d65df6461f5c41a27eef687ef53e28f0442c67915972aacf8a7"}
Nov 01 03:35:22.056153 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.056110    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"
Nov 01 03:35:22.058784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.058721    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj" event={"ID":"255a94a8-506a-4c39-b9c3-82299a68e6d6","Type":"ContainerDied","Data":"886bcce8d4b69d0f08fe9a1ecd76ccafaad08cdc162f3f2bb5e91dd5f919e7a0"}
Nov 01 03:35:22.058784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.058717    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"
Nov 01 03:35:22.065558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.065531    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"
Nov 01 03:35:22.065719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.065551    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj" event={"ID":"678ba707-c2e0-4a8a-bcb3-33390a9441bd","Type":"ContainerDied","Data":"313d8d4f18d6c03a26f83863ebf08416a2a076a219a485e1204b6bb580c8f75b"}
Nov 01 03:35:22.067447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.067422    4892 generic.go:356] "Generic (PLEG): container finished" podID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerID="a9144291f5b13cacc7631e69b36c8588151d53623e315e2fef4fbdad375e889d" exitCode=0
Nov 01 03:35:22.067602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.067519    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" event={"ID":"f5886cdb-de25-4b50-a6e0-adb1e826c943","Type":"ContainerDied","Data":"a9144291f5b13cacc7631e69b36c8588151d53623e315e2fef4fbdad375e889d"}
Nov 01 03:35:22.071470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.070783    4892 generic.go:356] "Generic (PLEG): container finished" podID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerID="2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c" exitCode=0
Nov 01 03:35:22.071470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.070848    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" event={"ID":"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5","Type":"ContainerDied","Data":"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"}
Nov 01 03:35:22.071470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.070919    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn" event={"ID":"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5","Type":"ContainerDied","Data":"e21b48d749697c95bab5e9ebe12d1e78d6f1d99749a456ae3c4ff8b08dd7dd2e"}
Nov 01 03:35:22.071470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.071066    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"
Nov 01 03:35:22.074556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.074536    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:35:22.076953 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.076929    4892 scope.go:122] "RemoveContainer" containerID="299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"
Nov 01 03:35:22.077452 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:22.077417    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17\": container with ID starting with 299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17 not found: ID does not exist" containerID="299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"
Nov 01 03:35:22.077560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.077460    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17"} err="failed to get container status \"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17\": rpc error: code = NotFound desc = could not find container \"299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17\": container with ID starting with 299d66a27b3149fce5a95d1452006df3f4ae1ea64d8272bfe0337e3cb92a0f17 not found: ID does not exist"
Nov 01 03:35:22.077560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.077481    4892 scope.go:122] "RemoveContainer" containerID="a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"
Nov 01 03:35:22.091847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.091793    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.092105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.091888    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.092105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092085    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092160    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092244    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092316    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092365    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092397    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.092449    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\") pod \"3ea25d44-8cda-466d-9299-50d80ce9f04f\" (UID: \"3ea25d44-8cda-466d-9299-50d80ce9f04f\") "
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.093356    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.093553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.093511    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.095564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.095452    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.095564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.095514    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:22.097332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.097303    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.098408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.098386    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.113918 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.101814    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "kube-api-access-4pxz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.120169    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.121658    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo" pod "3ea25d44-8cda-466d-9299-50d80ce9f04f" (UID: "3ea25d44-8cda-466d-9299-50d80ce9f04f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.123208    4892 scope.go:122] "RemoveContainer" containerID="a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:22.123607    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94\": container with ID starting with a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94 not found: ID does not exist" containerID="a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.123639    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94"} err="failed to get container status \"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94\": rpc error: code = NotFound desc = could not find container \"a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94\": container with ID starting with a58c7371963a064945b35635f2ba53714c554fd0e3015cb7c99460717f760d94 not found: ID does not exist"
Nov 01 03:35:22.128009 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.123661    4892 scope.go:122] "RemoveContainer" containerID="0427b9f32dd3a3946a7297044dfc66d4e3cc84c0247cc7e07108099289484f44"
Nov 01 03:35:22.135997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.135951    4892 scope.go:122] "RemoveContainer" containerID="c38397b455da33afaeef81923c7cfbcb768260bd26f355d15e1d28fbeea01246"
Nov 01 03:35:22.146469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.146441    4892 scope.go:122] "RemoveContainer" containerID="f647feda9d1a75ab8523fc5bb39bca77c23ff46ae9f4040202f849a206fc869e"
Nov 01 03:35:22.156719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.156696    4892 scope.go:122] "RemoveContainer" containerID="2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"
Nov 01 03:35:22.167349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.167323    4892 scope.go:122] "RemoveContainer" containerID="2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"
Nov 01 03:35:22.167710 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:22.167685    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c\": container with ID starting with 2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c not found: ID does not exist" containerID="2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"
Nov 01 03:35:22.167778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.167721    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c"} err="failed to get container status \"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c\": rpc error: code = NotFound desc = could not find container \"2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c\": container with ID starting with 2c0d5719ca8cd98ea8d1d62f012dce1b4021c107a573c1ae12bfff74decf1e8c not found: ID does not exist"
Nov 01 03:35:22.194026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.193921    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.194026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.193989    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194036    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194065    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194104    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194131    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194159    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.194281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194215    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194283    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194310    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194358    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194394    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194428    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194454    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194459    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194503    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.194590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194539    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194594    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194617    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194665    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194737    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194831    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194874    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194945    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194968    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195062    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195095    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.195129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195127    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.195693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195168    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\") pod \"ca4e8f78-70a6-449d-9dff-86629bb521b4\" (UID: \"ca4e8f78-70a6-449d-9dff-86629bb521b4\") "
Nov 01 03:35:22.195693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195228    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\") pod \"f5886cdb-de25-4b50-a6e0-adb1e826c943\" (UID: \"f5886cdb-de25-4b50-a6e0-adb1e826c943\") "
Nov 01 03:35:22.195693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195264    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\") pod \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\" (UID: \"46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5\") "
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196127    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196154    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196172    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196189    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ea25d44-8cda-466d-9299-50d80ce9f04f-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196205    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4pxz7\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-kube-api-access-4pxz7\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196221    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196235    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196249    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196264    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196279    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196295    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196310    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ea25d44-8cda-466d-9299-50d80ce9f04f-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.196839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196326    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ea25d44-8cda-466d-9299-50d80ce9f04f-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.194974    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195206    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195317    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.195380    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196601    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196635    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.197384    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.197428    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.196939    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.197312    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.197594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.197491    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.198267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.197920    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.198267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198129    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:22.198267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198242    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.198441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198266    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "kube-api-access-rs5mv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.198441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198337    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "kube-api-access-fjg7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.198441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198354    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.198794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.198763    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "kube-api-access-b7dcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.199093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.199066    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo" pod "ca4e8f78-70a6-449d-9dff-86629bb521b4" (UID: "ca4e8f78-70a6-449d-9dff-86629bb521b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:22.199200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.199073    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:35:22.199655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.199635    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.199744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.199705    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token" pod "f5886cdb-de25-4b50-a6e0-adb1e826c943" (UID: "f5886cdb-de25-4b50-a6e0-adb1e826c943"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:35:22.199744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.199731    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy" pod "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" (UID: "46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:35:22.297940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.297896    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298022    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298037    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298046    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298055    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298064    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca4e8f78-70a6-449d-9dff-86629bb521b4-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298073    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298084    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298094    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298107    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca4e8f78-70a6-449d-9dff-86629bb521b4-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298118    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298126    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298134    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5886cdb-de25-4b50-a6e0-adb1e826c943-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298144    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7dcs\" (UniqueName: \"kubernetes.io/projected/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-kube-api-access-b7dcs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298153    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298161    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298171    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298184    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjg7p\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-kube-api-access-fjg7p\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298195    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs5mv\" (UniqueName: \"kubernetes.io/projected/f5886cdb-de25-4b50-a6e0-adb1e826c943-kube-api-access-rs5mv\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298203    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5886cdb-de25-4b50-a6e0-adb1e826c943-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298219    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298227    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca4e8f78-70a6-449d-9dff-86629bb521b4-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298235    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.298639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.298244    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5886cdb-de25-4b50-a6e0-adb1e826c943-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:35:22.300404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.300381    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:22.336668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.336633    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:35:22.340005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.339965    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-rtvxj"]
Nov 01 03:35:22.444126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.444040    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:35:22.453099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.453060    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-nb69w"]
Nov 01 03:35:22.514222 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.514183    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:22.561226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.561191    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:35:22.567736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.567699    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-8mwmj"]
Nov 01 03:35:22.915021 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.914995    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:35:22.923917 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.923889    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:35:22.991595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.991556    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:35:22.994691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:22.994661    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-gdjqn"]
Nov 01 03:35:23.079612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.079520    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" event={"ID":"dbb72c63-aa0d-4bc1-977f-96272a4560b3","Type":"ContainerStarted","Data":"92d3b0e78ebf82a1614cceefad3ddb9a1d01a18da5650e6c0439d894215c4d43"}
Nov 01 03:35:23.085417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.085384    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9" event={"ID":"f5886cdb-de25-4b50-a6e0-adb1e826c943","Type":"ContainerDied","Data":"177635b6508e64d322a6f3ab982b5c5faad29c1558e2e427a4d41dabdf6c9069"}
Nov 01 03:35:23.085566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.085450    4892 scope.go:122] "RemoveContainer" containerID="a9144291f5b13cacc7631e69b36c8588151d53623e315e2fef4fbdad375e889d"
Nov 01 03:35:23.085566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.085396    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"
Nov 01 03:35:23.101157 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.101129    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:35:23.108216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.108190    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-hbz7l"]
Nov 01 03:35:23.219375 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.219341    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:35:23.223795 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.223758    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-6t9l4"]
Nov 01 03:35:23.335197 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.335093    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:35:23.341046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.341016    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-e8c789-openshift-default-774d997df-bbrp9"]
Nov 01 03:35:23.706943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.706848    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="255a94a8-506a-4c39-b9c3-82299a68e6d6" path="/var/lib/kubelet/pods/255a94a8-506a-4c39-b9c3-82299a68e6d6/volumes"
Nov 01 03:35:23.707650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.707628    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" path="/var/lib/kubelet/pods/3ea25d44-8cda-466d-9299-50d80ce9f04f/volumes"
Nov 01 03:35:23.708284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.708264    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" path="/var/lib/kubelet/pods/46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5/volumes"
Nov 01 03:35:23.709061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.709045    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c06d5e6-550f-4454-a6a9-6f5ec557c253" path="/var/lib/kubelet/pods/5c06d5e6-550f-4454-a6a9-6f5ec557c253/volumes"
Nov 01 03:35:23.709471 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.709458    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="678ba707-c2e0-4a8a-bcb3-33390a9441bd" path="/var/lib/kubelet/pods/678ba707-c2e0-4a8a-bcb3-33390a9441bd/volumes"
Nov 01 03:35:23.709882 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.709870    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" path="/var/lib/kubelet/pods/ca4e8f78-70a6-449d-9dff-86629bb521b4/volumes"
Nov 01 03:35:23.710500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:23.710484    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" path="/var/lib/kubelet/pods/f5886cdb-de25-4b50-a6e0-adb1e826c943/volumes"
Nov 01 03:35:24.101703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:24.101660    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" event={"ID":"dbb72c63-aa0d-4bc1-977f-96272a4560b3","Type":"ContainerStarted","Data":"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"}
Nov 01 03:35:24.102231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:24.102072    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:24.220286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:24.220220    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" podStartSLOduration=2.339223529 podStartE2EDuration="3.220200012s" totalImagesPullingTime="880.976483ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:35:24.219011641 +0000 UTC m=+1690.634911169" watchObservedRunningTime="2026-11-01 03:35:24.220200012 +0000 UTC m=+1690.636099544"
Nov 01 03:35:27.611376 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:27.611340    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:35:30.118302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:30.118269    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:35:46.510532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.510493    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:35:46.511386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511368    4892 cpu_manager.go:414] "containerMap: removing container" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy"
Nov 01 03:35:46.511457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511388    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy"
Nov 01 03:35:46.511457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511408    4892 cpu_manager.go:414] "containerMap: removing container" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy"
Nov 01 03:35:46.511457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511415    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy"
Nov 01 03:35:46.511457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511440    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy"
Nov 01 03:35:46.511457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511448    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511465    4892 cpu_manager.go:414] "containerMap: removing container" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511472    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511580    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ea25d44-8cda-466d-9299-50d80ce9f04f" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511600    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca4e8f78-70a6-449d-9dff-86629bb521b4" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511609    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="46f41dfa-56ab-48e0-9ebd-f0d5e220ccf5" containerName="istio-proxy"
Nov 01 03:35:46.511624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.511620    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5886cdb-de25-4b50-a6e0-adb1e826c943" containerName="istio-proxy"
Nov 01 03:35:46.515574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.515549    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.526477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.526453    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.526560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.526484    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w68zs\" (UniqueName: \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.526560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.526516    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.526710 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.526692    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.619674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.619637    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-686153-serving-cert\""
Nov 01 03:35:46.622914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.622878    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:35:46.628181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.628158    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.628308 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.628237    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.628373 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:46.628325    4892 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-serving-cert: secret "maas-api-e2e-shared-b-w4-686153-serving-cert" not found
Nov 01 03:35:46.628432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.628379    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.628432 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:46.628410    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls podName:97b2f1c3-b3a0-48cd-99d7-2fb944317425 nodeName:}" failed. No retries permitted until 2026-11-01 03:35:47.128385763 +0000 UTC m=+1713.544285282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls") pod "maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" (UID: "97b2f1c3-b3a0-48cd-99d7-2fb944317425") : secret "maas-api-e2e-shared-b-w4-686153-serving-cert" not found
Nov 01 03:35:46.628545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.628450    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w68zs\" (UniqueName: \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.629105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.629077    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.631024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.630993    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:46.743582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:46.743550    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w68zs\" (UniqueName: \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:47.148019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.147957    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:47.150631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.150598    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:47.253873 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.253824    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:35:47.258190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.258167    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.346035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.345971    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:35:47.351119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351081    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351147    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvgm\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351216    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351261    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351295 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351291    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351326    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351472    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351630    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.351782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.351666    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.387323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.385631    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f2383c-openshift-default-dockercfg-bm652\""
Nov 01 03:35:47.432053 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.431307    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:47.477223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477188    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477250    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477339    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477386    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvgm\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477449    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477493    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477523    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477563 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477559    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.477770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.477713    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.478561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.478533    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.478821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.478798    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.479085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.479065    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.480293 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.479701    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.480507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.480485    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.482820 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.482784    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.483811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.483790    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.558660 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.558613    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.567482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.567443    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvgm\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\") pod \"e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") " pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.580271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.580232    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:47.868380 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.868355    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:35:47.871425 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:35:47.871390    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97b2f1c3_b3a0_48cd_99d7_2fb944317425.slice/crio-5540ac6c48d0759686cef01a8bdf050c314b80c82a7f6ac9df2819cebe870d90 WatchSource:0}: Error finding container 5540ac6c48d0759686cef01a8bdf050c314b80c82a7f6ac9df2819cebe870d90: Status 404 returned error can't find the container with id 5540ac6c48d0759686cef01a8bdf050c314b80c82a7f6ac9df2819cebe870d90
Nov 01 03:35:47.996391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:47.996358    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:35:47.998514 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:35:47.998482    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod387d4451_4574_458f_8f1c_230507800e48.slice/crio-c71e74c23dc29090183539881091db38e8a292f9638dddb1e02f25903d122c1a WatchSource:0}: Error finding container c71e74c23dc29090183539881091db38e8a292f9638dddb1e02f25903d122c1a: Status 404 returned error can't find the container with id c71e74c23dc29090183539881091db38e8a292f9638dddb1e02f25903d122c1a
Nov 01 03:35:48.001128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.001092    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:35:48.001240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.001177    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:35:48.001240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.001222    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:35:48.309809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.309769    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" event={"ID":"97b2f1c3-b3a0-48cd-99d7-2fb944317425","Type":"ContainerStarted","Data":"5540ac6c48d0759686cef01a8bdf050c314b80c82a7f6ac9df2819cebe870d90"}
Nov 01 03:35:48.311945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.311914    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerStarted","Data":"cfc3e8364730614c75e403e6a4e37e15208071d9b8884ed46928ad0d75e760cf"}
Nov 01 03:35:48.312114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.311970    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerStarted","Data":"c71e74c23dc29090183539881091db38e8a292f9638dddb1e02f25903d122c1a"}
Nov 01 03:35:48.438039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.437957    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podStartSLOduration=1.437939582 podStartE2EDuration="1.437939582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:35:48.43234955 +0000 UTC m=+1714.848249079" watchObservedRunningTime="2026-11-01 03:35:48.437939582 +0000 UTC m=+1714.853839113"
Nov 01 03:35:48.581303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:48.581203    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:49.321285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:49.321247    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" event={"ID":"97b2f1c3-b3a0-48cd-99d7-2fb944317425","Type":"ContainerStarted","Data":"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"}
Nov 01 03:35:49.321498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:49.321332    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:49.442939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:49.442874    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" podStartSLOduration=2.540976427 podStartE2EDuration="3.442853922s" totalImagesPullingTime="901.877495ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:35:49.442557684 +0000 UTC m=+1715.858457214" watchObservedRunningTime="2026-11-01 03:35:49.442853922 +0000 UTC m=+1715.858753456"
Nov 01 03:35:49.593180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:49.593083    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:50.329049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:50.329016    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:50.329912 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:50.329892    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:35:55.335042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:35:55.334952    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:35:57.627781 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:35:57.627745    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:36:13.859051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.859015    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:36:13.863297 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.863275    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.967589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967550    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.967876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967850    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.967876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967628    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fdbdfefd\"/\"openshift-service-ca.crt\""
Nov 01 03:36:13.968121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967692    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fdbdfefd\"/\"default-dockercfg-gkmjh\""
Nov 01 03:36:13.968121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967628    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-fdbdfefd\"/\"kube-root-ca.crt\""
Nov 01 03:36:13.968121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967858    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-fdbdfefd\"/\"e2e-lifecycle-model-fdbdfefd-kserve-self-signed-certs\""
Nov 01 03:36:13.968121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.967972    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.968121 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.968046    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.968420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.968150    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.968420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.968223    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs64l\" (UniqueName: \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:13.972098 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:13.972061    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:36:14.069834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.069777    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.069834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.069823    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.069974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.069885    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.069974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.069946    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs64l\" (UniqueName: \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.070308 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.070284    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.070381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.070306    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.070381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.070320    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.070381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.070355    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.070538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.070378    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.076052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.076023    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.076298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.076263    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.139875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.139798    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs64l\" (UniqueName: \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\") pod \"e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") " pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.175870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.175828    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:14.575950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:14.575913    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:36:14.577296 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:14.577269    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90889b05_c6c0_4f01_b3e2_5d95cc1ab4bb.slice/crio-e26edf6236df1712f6c8dbd89525f0976441fb7e5adfeefe8102b37eafac01dc WatchSource:0}: Error finding container e26edf6236df1712f6c8dbd89525f0976441fb7e5adfeefe8102b37eafac01dc: Status 404 returned error can't find the container with id e26edf6236df1712f6c8dbd89525f0976441fb7e5adfeefe8102b37eafac01dc
Nov 01 03:36:15.507673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:15.507631    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" event={"ID":"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb","Type":"ContainerStarted","Data":"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"}
Nov 01 03:36:15.507673 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:15.507681    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" event={"ID":"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb","Type":"ContainerStarted","Data":"e26edf6236df1712f6c8dbd89525f0976441fb7e5adfeefe8102b37eafac01dc"}
Nov 01 03:36:15.627254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:15.627193    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" podStartSLOduration=2.6271796739999997 podStartE2EDuration="2.627179674s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:36:15.624327705 +0000 UTC m=+1742.040227236" watchObservedRunningTime="2026-11-01 03:36:15.627179674 +0000 UTC m=+1742.043079205"
Nov 01 03:36:16.176669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:16.176630    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:24.176845 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:24.176804    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:24.189648 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:24.189615    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:24.584635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:24.584605    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:36:27.615063 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:36:27.615019    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:36:38.258925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.258886    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:36:38.263172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.263149    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292594    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292645    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292672    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqff8\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292722    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292784    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.292999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.292924    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.293444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.293146    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.293444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.293194    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.293444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.293243    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.394002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.393542    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-634332-openshift-default-dockercfg-nwhq2\""
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456146    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456231    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456280    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456327    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456477    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456504    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456527    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqff8\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456583    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.456713    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.457594    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.457847    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.457880    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.457896    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.458111    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.459456    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.464078    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.469997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.464574    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.542652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.542531    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.548488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.548457    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqff8\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\") pod \"e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") " pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:38.578267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:38.578221    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:39.060266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.060228    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:36:39.067440 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:39.067411    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663bb7b3_36a9_427e_8175_fe1b2978a3db.slice/crio-4a1e263e2c34d8223c1878bb2ae0cf8d5f22e1c6f888d329c4bb2f8ec9100b83 WatchSource:0}: Error finding container 4a1e263e2c34d8223c1878bb2ae0cf8d5f22e1c6f888d329c4bb2f8ec9100b83: Status 404 returned error can't find the container with id 4a1e263e2c34d8223c1878bb2ae0cf8d5f22e1c6f888d329c4bb2f8ec9100b83
Nov 01 03:36:39.070477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.070433    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:36:39.070586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.070530    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:36:39.070660 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.070581    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:36:39.837276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.837238    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerStarted","Data":"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"}
Nov 01 03:36:39.837698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.837301    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerStarted","Data":"4a1e263e2c34d8223c1878bb2ae0cf8d5f22e1c6f888d329c4bb2f8ec9100b83"}
Nov 01 03:36:39.970712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:39.970651    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podStartSLOduration=1.970632602 podStartE2EDuration="1.970632602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:36:39.965702564 +0000 UTC m=+1766.381602116" watchObservedRunningTime="2026-11-01 03:36:39.970632602 +0000 UTC m=+1766.386532132"
Nov 01 03:36:40.587938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.581126    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:40.589177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.589140    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:40.888107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.888039    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:36:40.910160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.909709    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:40.910160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.909762    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:36:40.910160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.909832    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:36:40.911062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.910560    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:40.964747 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.964713    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w6-7cd000\""
Nov 01 03:36:40.964962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:40.964838    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w6-7cd000-dockercfg-49zgx\""
Nov 01 03:36:41.035651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.035621    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.035838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.035813    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khrqn\" (UniqueName: \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.137709 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.137676    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khrqn\" (UniqueName: \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.137921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.137884    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.138649 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.138599    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.199359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.199325    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khrqn\" (UniqueName: \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\") pod \"payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.236304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.236266    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:41.628570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.628540    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:36:41.629757 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:41.629730    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd234b547_be01_4cc0_bab9_d0f57f9540f4.slice/crio-066858d80e3e0ee4bdb68e739d921158de2781177575272f916edc21985d5059 WatchSource:0}: Error finding container 066858d80e3e0ee4bdb68e739d921158de2781177575272f916edc21985d5059: Status 404 returned error can't find the container with id 066858d80e3e0ee4bdb68e739d921158de2781177575272f916edc21985d5059
Nov 01 03:36:41.900619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.900518    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" event={"ID":"d234b547-be01-4cc0-bab9-d0f57f9540f4","Type":"ContainerStarted","Data":"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"}
Nov 01 03:36:41.900619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.900580    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" event={"ID":"d234b547-be01-4cc0-bab9-d0f57f9540f4","Type":"ContainerStarted","Data":"066858d80e3e0ee4bdb68e739d921158de2781177575272f916edc21985d5059"}
Nov 01 03:36:41.901090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:41.900882    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:42.018647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:42.018589    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" podStartSLOduration=2.018573766 podStartE2EDuration="2.018573766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:36:42.015912261 +0000 UTC m=+1768.431811802" watchObservedRunningTime="2026-11-01 03:36:42.018573766 +0000 UTC m=+1768.434473295"
Nov 01 03:36:48.532261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.532223    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:36:48.543665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.543637    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.593832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.593795    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:36:48.638705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.638670    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-634332\""
Nov 01 03:36:48.638950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.638776    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-634332-dockercfg-l2h4q\""
Nov 01 03:36:48.675793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.675761    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7zbw\" (UniqueName: \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.675956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.675935    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.697264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.697224    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:36:48.703355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.703300    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:48.763714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.763265    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:36:48.834557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.834166    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.836199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.836173    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlb5v\" (UniqueName: \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:48.837027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.835852    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.837181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.837087    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7zbw\" (UniqueName: \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.837394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.837373    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:48.898615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.898578    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7zbw\" (UniqueName: \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\") pod \"payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:48.939252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.939205    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlb5v\" (UniqueName: \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:48.939455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.939293    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:48.940052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:48.940019    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:49.000610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.000577    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlb5v\" (UniqueName: \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\") pod \"payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:49.040231 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.040197    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:49.173872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.173841    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:49.422415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.422383    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:36:49.423353 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:49.423326    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeea8e1eb_fbc4_4cb9_ae94_b090ddebe92b.slice/crio-40c461b5e10737a6dd5b513b870ad20f5619cf3e6a9d9b868badaf88a9f3949f WatchSource:0}: Error finding container 40c461b5e10737a6dd5b513b870ad20f5619cf3e6a9d9b868badaf88a9f3949f: Status 404 returned error can't find the container with id 40c461b5e10737a6dd5b513b870ad20f5619cf3e6a9d9b868badaf88a9f3949f
Nov 01 03:36:49.587468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.587439    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:36:49.588324 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:49.588296    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4b4ccee_c768_4685_a695_6dd5dd551519.slice/crio-fc663dc6ecb118f4e9f9cd43b1edfda83cb0eb2c38f389e1a6d2308e43779876 WatchSource:0}: Error finding container fc663dc6ecb118f4e9f9cd43b1edfda83cb0eb2c38f389e1a6d2308e43779876: Status 404 returned error can't find the container with id fc663dc6ecb118f4e9f9cd43b1edfda83cb0eb2c38f389e1a6d2308e43779876
Nov 01 03:36:49.953645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.953606    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" event={"ID":"d4b4ccee-c768-4685-a695-6dd5dd551519","Type":"ContainerStarted","Data":"558cdc6cd28622618c201d66a92bc530005ecfa2f65d9e2725b7e5c47cc9e921"}
Nov 01 03:36:49.953843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.953667    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" event={"ID":"d4b4ccee-c768-4685-a695-6dd5dd551519","Type":"ContainerStarted","Data":"fc663dc6ecb118f4e9f9cd43b1edfda83cb0eb2c38f389e1a6d2308e43779876"}
Nov 01 03:36:49.953843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.953698    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:36:49.955335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.955307    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" event={"ID":"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b","Type":"ContainerStarted","Data":"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"}
Nov 01 03:36:49.955458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.955358    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" event={"ID":"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b","Type":"ContainerStarted","Data":"40c461b5e10737a6dd5b513b870ad20f5619cf3e6a9d9b868badaf88a9f3949f"}
Nov 01 03:36:49.955458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:49.955380    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:36:50.072970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:50.072914    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" podStartSLOduration=2.072900189 podStartE2EDuration="2.072900189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:36:50.069650507 +0000 UTC m=+1776.485550036" watchObservedRunningTime="2026-11-01 03:36:50.072900189 +0000 UTC m=+1776.488799717"
Nov 01 03:36:50.189757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:50.189697    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" podStartSLOduration=2.189672968 podStartE2EDuration="2.189672968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:36:50.187221534 +0000 UTC m=+1776.603121064" watchObservedRunningTime="2026-11-01 03:36:50.189672968 +0000 UTC m=+1776.605572514"
Nov 01 03:36:52.911218 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:52.911183    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:36:57.614161 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:36:57.614119    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:36:58.647072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.647036    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:36:58.651201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.651182    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.704111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.704082    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-f2383c-serving-cert\""
Nov 01 03:36:58.757996 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.757953    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:36:58.795424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.795393    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.795597 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.795447    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrvns\" (UniqueName: \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.795674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.795656    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.795717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.795696    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.897482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.897382    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.897482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.897448    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrvns\" (UniqueName: \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.897677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.897631    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.897723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.897678    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.898491 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.898466    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.900197 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.900170    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.900197 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.900179    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.959290 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.959250    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrvns\" (UniqueName: \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\") pod \"maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:58.966079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:58.966052    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:36:59.396493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:36:59.396457    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:36:59.398372 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:36:59.398333    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod243c6dcb_5b19_42c9_a517_be3e3cb100a3.slice/crio-51e399986ed6c20b2648da217fb60dfce660ead589520e1c3014098be6600ddd WatchSource:0}: Error finding container 51e399986ed6c20b2648da217fb60dfce660ead589520e1c3014098be6600ddd: Status 404 returned error can't find the container with id 51e399986ed6c20b2648da217fb60dfce660ead589520e1c3014098be6600ddd
Nov 01 03:37:00.027353 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:00.027312    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" event={"ID":"243c6dcb-5b19-42c9-a517-be3e3cb100a3","Type":"ContainerStarted","Data":"51e399986ed6c20b2648da217fb60dfce660ead589520e1c3014098be6600ddd"}
Nov 01 03:37:00.437653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:00.437612    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:37:00.437917 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:00.437891    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" containerID="cri-o://30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd" gracePeriod=30
Nov 01 03:37:00.965636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:00.965606    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:37:00.965859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:00.965680    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:37:01.038825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:01.038785    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" event={"ID":"243c6dcb-5b19-42c9-a517-be3e3cb100a3","Type":"ContainerStarted","Data":"31ee7372441d373c6ccf151adced0f212640503414ebf264c9dfeb4470fd8bf5"}
Nov 01 03:37:01.039235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:01.039126    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:37:01.071004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:01.069938    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:37:01.071004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:01.070210    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" containerID="cri-o://8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6" gracePeriod=30
Nov 01 03:37:01.323782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:01.323725    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" podStartSLOduration=2.306413953 podStartE2EDuration="3.323710111s" totalImagesPullingTime="1.017296158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:37:01.320938972 +0000 UTC m=+1787.736838498" watchObservedRunningTime="2026-11-01 03:37:01.323710111 +0000 UTC m=+1787.739609639"
Nov 01 03:37:02.023734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.023693    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:37:02.036355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.036319    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.083935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.083902    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:37:02.130136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.130047    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 01 03:37:02.130598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.130498    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8w9bn\""
Nov 01 03:37:02.130703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.130520    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 01 03:37:02.130788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.130563    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 01 03:37:02.158748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.158714    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.158930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.158769    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.159046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.158945    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.159046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.159011    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.159163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.159110    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.159350 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.159333    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.159412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.159361    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbsf2\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.260718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.260670    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.260918 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.260734    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.260918 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.260860    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.260918 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.260915    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.261124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.260951    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.261124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.261102    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.261224 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.261163    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbsf2\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.261686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.261652    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.263829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.263796    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.264058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.264033    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.264147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.264077    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.264195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.264178    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.322035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.321935    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.322289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.322270    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbsf2\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\") pod \"istiod-openshift-gateway-7ddd9cfdc8-44mwv\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.363123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.363089    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:02.831883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.831855    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:37:02.832823 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:37:02.832797    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08c0f6c_80eb_49ea_b362_07767a84306c.slice/crio-cd6e94be8f5943325ff375f6f49ced926df268f3dc30180771cd2c7f6de09226 WatchSource:0}: Error finding container cd6e94be8f5943325ff375f6f49ced926df268f3dc30180771cd2c7f6de09226: Status 404 returned error can't find the container with id cd6e94be8f5943325ff375f6f49ced926df268f3dc30180771cd2c7f6de09226
Nov 01 03:37:02.970949 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.970909    4892 patch_prober.go:29] interesting pod/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:37:02.971155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:02.970990    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:37:03.054831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:03.054790    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" event={"ID":"e08c0f6c-80eb-49ea-b362-07767a84306c","Type":"ContainerStarted","Data":"cd6e94be8f5943325ff375f6f49ced926df268f3dc30180771cd2c7f6de09226"}
Nov 01 03:37:04.161413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:04.161369    4892 patch_prober.go:29] interesting pod/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:37:04.161821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:04.161426    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:37:05.751765 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.751731    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:37:05.811753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.811720    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.811955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.811795    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.811955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.811830    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.811955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.811891    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.811955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.811930    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.812288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812116    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.812288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812161    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:05.812288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812216    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.812288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812244    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.812288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812250    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:05.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812303    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\") pod \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\" (UID: \"6a5a51fb-e4f0-42bb-b501-00769e91ed80\") "
Nov 01 03:37:05.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812354    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:37:05.812545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812469    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:05.812708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.812680    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:05.813107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.813079    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.813202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.813113    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.813202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.813128    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.813202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.813142    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.813202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.813157    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.814672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.814647    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:37:05.814672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.814658    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:37:05.815693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.815670    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "kube-api-access-kmq9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:37:05.816099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.816080    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy" pod "6a5a51fb-e4f0-42bb-b501-00769e91ed80" (UID: "6a5a51fb-e4f0-42bb-b501-00769e91ed80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:05.914583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.914549    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmq9j\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-kube-api-access-kmq9j\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.914583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.914581    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.914812 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.914595    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:05.914812 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:05.914607    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a5a51fb-e4f0-42bb-b501-00769e91ed80-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.082585 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.082539    4892 generic.go:356] "Generic (PLEG): container finished" podID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerID="30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd" exitCode=0
Nov 01 03:37:06.082886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.082618    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"
Nov 01 03:37:06.082886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.082633    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" event={"ID":"6a5a51fb-e4f0-42bb-b501-00769e91ed80","Type":"ContainerDied","Data":"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"}
Nov 01 03:37:06.082886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.082688    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n" event={"ID":"6a5a51fb-e4f0-42bb-b501-00769e91ed80","Type":"ContainerDied","Data":"d53f9b569ff0578b9708f781b29c387510143f9c12c7a231df58522e923241ae"}
Nov 01 03:37:06.082886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.082749    4892 scope.go:122] "RemoveContainer" containerID="30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"
Nov 01 03:37:06.096465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.096442    4892 scope.go:122] "RemoveContainer" containerID="30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"
Nov 01 03:37:06.096897 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:37:06.096859    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd\": container with ID starting with 30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd not found: ID does not exist" containerID="30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"
Nov 01 03:37:06.097086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.096913    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd"} err="failed to get container status \"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd\": rpc error: code = NotFound desc = could not find container \"30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd\": container with ID starting with 30aae793acb8965f53f6f95d2c89ed9b598fc24b30229e9b2ca4bd206a7903dd not found: ID does not exist"
Nov 01 03:37:06.268827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.268785    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:37:06.275448 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.275420    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-txp9n"]
Nov 01 03:37:06.502077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.502050    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.537919    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.537970    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538050    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538104    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538235    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538319    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538499    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538548    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.538579    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\") pod \"119d01d7-b601-495c-8945-90ddc2110f5d\" (UID: \"119d01d7-b601-495c-8945-90ddc2110f5d\") "
Nov 01 03:37:06.539588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.539554    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:37:06.540312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.539872    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:06.540510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.540424    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:06.540510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.540503    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:06.547070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.540850    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:06.547070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.540874    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "kube-api-access-k6f4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:37:06.560274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.552724    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:37:06.560274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.552737    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:37:06.560274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.555465    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token" pod "119d01d7-b601-495c-8945-90ddc2110f5d" (UID: "119d01d7-b601-495c-8945-90ddc2110f5d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:37:06.641247 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641206    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641247 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641243    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641257    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641271    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6f4b\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-kube-api-access-k6f4b\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641284    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/119d01d7-b601-495c-8945-90ddc2110f5d-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641296    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641308    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/119d01d7-b601-495c-8945-90ddc2110f5d-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641322    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/119d01d7-b601-495c-8945-90ddc2110f5d-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:06.641481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:06.641335    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/119d01d7-b601-495c-8945-90ddc2110f5d-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:37:07.056029 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.055965    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:37:07.096077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.096036    4892 generic.go:356] "Generic (PLEG): container finished" podID="119d01d7-b601-495c-8945-90ddc2110f5d" containerID="8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6" exitCode=0
Nov 01 03:37:07.096270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.096109    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"
Nov 01 03:37:07.096270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.096122    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" event={"ID":"119d01d7-b601-495c-8945-90ddc2110f5d","Type":"ContainerDied","Data":"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"}
Nov 01 03:37:07.096270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.096179    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh" event={"ID":"119d01d7-b601-495c-8945-90ddc2110f5d","Type":"ContainerDied","Data":"eedfbf4028d22cb7c75e9725a74bb2911010a0167b5ec42247db369cc73f4c03"}
Nov 01 03:37:07.096270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.096211    4892 scope.go:122] "RemoveContainer" containerID="8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"
Nov 01 03:37:07.114359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.114328    4892 scope.go:122] "RemoveContainer" containerID="8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"
Nov 01 03:37:07.115035 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:37:07.114955    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6\": container with ID starting with 8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6 not found: ID does not exist" containerID="8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"
Nov 01 03:37:07.115161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.115050    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6"} err="failed to get container status \"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6\": rpc error: code = NotFound desc = could not find container \"8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6\": container with ID starting with 8f3323165f73f54b9c0cffa3549d22c3881d9bbd3230500e0e40a47f9b6a3ed6 not found: ID does not exist"
Nov 01 03:37:07.349450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.349320    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:37:07.353139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.353106    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mnxhh"]
Nov 01 03:37:07.490201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.490161    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:37:07.490580 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.490234    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:37:07.702210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.702133    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" path="/var/lib/kubelet/pods/119d01d7-b601-495c-8945-90ddc2110f5d/volumes"
Nov 01 03:37:07.702744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:07.702729    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" path="/var/lib/kubelet/pods/6a5a51fb-e4f0-42bb-b501-00769e91ed80/volumes"
Nov 01 03:37:08.107286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:08.107236    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" event={"ID":"e08c0f6c-80eb-49ea-b362-07767a84306c","Type":"ContainerStarted","Data":"b9f73d025ff9648df9d15fa754ebfe0956affe3d28b3245ecf1086c4611aacaa"}
Nov 01 03:37:08.107757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:08.107314    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:08.109972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:08.109943    4892 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-44mwv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:37:08.110128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:08.110006    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:37:08.247850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:08.247656    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" podStartSLOduration=2.594056164 podStartE2EDuration="7.247641717s" totalImagesPullingTime="4.653585553s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:37:08.246329803 +0000 UTC m=+1794.662229314" watchObservedRunningTime="2026-11-01 03:37:08.247641717 +0000 UTC m=+1794.663541246"
Nov 01 03:37:09.124027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:09.122504    4892 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-44mwv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:37:09.124027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:09.122581    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:37:12.118139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:12.118102    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:37:13.920640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:13.920614    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:37:13.923136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:13.923114    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:37:27.613061 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:37:27.613020    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:37:28.333437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.333401    4892 generic.go:356] "Generic (PLEG): container finished" podID="387d4451-4574-458f-8f1c-230507800e48" containerID="cfc3e8364730614c75e403e6a4e37e15208071d9b8884ed46928ad0d75e760cf" exitCode=0
Nov 01 03:37:28.333657 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.333478    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerDied","Data":"cfc3e8364730614c75e403e6a4e37e15208071d9b8884ed46928ad0d75e760cf"}
Nov 01 03:37:28.333929 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.333914    4892 scope.go:122] "RemoveContainer" containerID="cfc3e8364730614c75e403e6a4e37e15208071d9b8884ed46928ad0d75e760cf"
Nov 01 03:37:28.335775 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.335747    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:37:28.335851 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.335820    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:37:28.335893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:28.335856    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:37:29.354540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:29.354502    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerStarted","Data":"068f29e332ece196b73aa912bd7d8da060418d1d1191b06f9f4dadbb34718cae"}
Nov 01 03:37:29.581032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:29.581001    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:37:29.581268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:29.581200    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:29.581354 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:29.581263    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:30.581177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:30.581143    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:30.581591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:30.581195    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:31.580860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:31.580827    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:31.581082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:31.580890    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:32.581110 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:32.581071    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:32.581545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:32.581135    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:33.580888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:33.580845    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:33.581116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:33.580913    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:34.580945 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:34.580906    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused" start-of-body=
Nov 01 03:37:34.581484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:34.581008    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.92:15021/healthz/ready\": dial tcp 10.134.0.92:15021: connect: connection refused"
Nov 01 03:37:35.329875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:35.329841    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:37:35.586106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:35.586027    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:37:35.587143 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:37:35.587123    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:37:57.613346 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:37:57.613309    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:38:27.614324 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:38:27.614282    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:38:41.932152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:41.932106    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 03:38:41.932654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:41.932216    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:38:42.901129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.901096    4892 generic.go:356] "Generic (PLEG): container finished" podID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerID="056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1" exitCode=0
Nov 01 03:38:42.901322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.901167    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerDied","Data":"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"}
Nov 01 03:38:42.901645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.901630    4892 scope.go:122] "RemoveContainer" containerID="056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"
Nov 01 03:38:42.903409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.903379    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:42.903529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.903464    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:42.903529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:42.903515    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:43.911230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:43.910281    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerStarted","Data":"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"}
Nov 01 03:38:44.579130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:44.579090    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:38:44.579370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:44.579348    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 01 03:38:44.579429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:44.579407    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 01 03:38:45.578718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:45.578682    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 01 03:38:45.579156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:45.578748    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 01 03:38:46.579016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:46.578971    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 01 03:38:46.579445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:46.579046    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 01 03:38:47.578675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:47.578632    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 01 03:38:47.578854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:47.578704    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 01 03:38:48.578700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:48.578657    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused" start-of-body=
Nov 01 03:38:48.579166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:48.578729    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.94:15021/healthz/ready\": dial tcp 10.134.0.94:15021: connect: connection refused"
Nov 01 03:38:49.583436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:49.583406    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:38:49.583854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:49.583832    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:38:49.584633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:49.584615    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:38:50.478050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478015    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:38:50.478714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478686    4892 cpu_manager.go:414] "containerMap: removing container" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy"
Nov 01 03:38:50.478714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478704    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy"
Nov 01 03:38:50.478950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478741    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy"
Nov 01 03:38:50.478950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478750    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy"
Nov 01 03:38:50.478950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478864    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a5a51fb-e4f0-42bb-b501-00769e91ed80" containerName="istio-proxy"
Nov 01 03:38:50.478950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.478876    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="119d01d7-b601-495c-8945-90ddc2110f5d" containerName="istio-proxy"
Nov 01 03:38:50.481251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.481234    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.520731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.520699    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.520897 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.520816    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.520897 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.520872    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.520911    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.520965    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.521018    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pflbt\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.521111    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.521159    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.521282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.521261    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.538382 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.538353    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-438641-openshift-default-dockercfg-4hlvf\""
Nov 01 03:38:50.592246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.592213    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:38:50.622672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622641    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622675    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622703    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622723    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622750    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pflbt\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622815    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.622935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622874    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.622950    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623077    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623142    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623441    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623520    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623693    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.623854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.623743    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.625687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.625659    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.626406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.626383    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.685396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.685343    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.686090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.686034    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pflbt\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:50.794086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:50.794053    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:51.272566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:51.272533    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:38:51.273414 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:38:51.273383    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod015c1cd8_f0e0_4ccf_8719_51b08452764e.slice/crio-407034ff19173ebb00b849556a3456c2c9b7f0b948914e4cfb0b54b88758ff15 WatchSource:0}: Error finding container 407034ff19173ebb00b849556a3456c2c9b7f0b948914e4cfb0b54b88758ff15: Status 404 returned error can't find the container with id 407034ff19173ebb00b849556a3456c2c9b7f0b948914e4cfb0b54b88758ff15
Nov 01 03:38:51.276277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:51.276247    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:51.276351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:51.276320    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:51.276394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:51.276381    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:38:52.013678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.013594    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" event={"ID":"015c1cd8-f0e0-4ccf-8719-51b08452764e","Type":"ContainerStarted","Data":"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"}
Nov 01 03:38:52.013678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.013648    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" event={"ID":"015c1cd8-f0e0-4ccf-8719-51b08452764e","Type":"ContainerStarted","Data":"407034ff19173ebb00b849556a3456c2c9b7f0b948914e4cfb0b54b88758ff15"}
Nov 01 03:38:52.141681 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.141616    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podStartSLOduration=2.1415951460000002 podStartE2EDuration="2.141595146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:38:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:38:52.140485042 +0000 UTC m=+1898.556384575" watchObservedRunningTime="2026-11-01 03:38:52.141595146 +0000 UTC m=+1898.557494677"
Nov 01 03:38:52.794674 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.794637    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:52.796214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.796183    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 01 03:38:52.796336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:52.796247    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 01 03:38:53.795398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:53.795251    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 01 03:38:53.795398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:53.795303    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 01 03:38:54.795311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:54.795278    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 01 03:38:54.795470 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:54.795328    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 01 03:38:55.795373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:55.795329    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 01 03:38:55.795592 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:55.795394    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 01 03:38:56.794576 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:56.794535    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused" start-of-body=
Nov 01 03:38:56.794756 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:56.794612    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.100:15021/healthz/ready\": dial tcp 10.134.0.100:15021: connect: connection refused"
Nov 01 03:38:57.616206 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:38:57.616168    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:38:57.799409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:57.799382    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:57.799705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:57.799682    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:38:57.800453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:38:57.800438    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:39:13.051355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:13.051299    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:39:18.052167 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:18.052095    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing" containerID="cri-o://75100b94b63229f56653a0069c34d7f8c376c3a8d6707fdec1487f046e33a1ab" gracePeriod=25
Nov 01 03:39:19.242602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:19.242561    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:39:27.622005 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:39:27.621955    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:39:29.242368 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:29.242329    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:39:35.854160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.854126    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:39:35.858402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.858382    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.955242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.955207    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:39:35.956041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956013    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956058    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956151    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956182    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956279    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956417    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956566 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956548    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkk58\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956596    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:35.956764 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:35.956724    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058017    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058150    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkk58\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058194    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058266    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058331    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058368    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058393    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058404    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058502    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058555    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058689    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058766    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.058962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058937    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.059068 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.058941    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.060780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.060754    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.060970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.060953    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.120391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.120313    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.120541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.120426    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkk58\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\") pod \"e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") " pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.172435 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.172399    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:36.618801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.618778    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:39:36.620042 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:39:36.620013    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61942bc5_8d1f_4fa9_afeb_6c4a051a4cc1.slice/crio-c270c29ff04945f8a9ee3a1930d2a15c434d0b05a98692b84b0e5117ffdffacd WatchSource:0}: Error finding container c270c29ff04945f8a9ee3a1930d2a15c434d0b05a98692b84b0e5117ffdffacd: Status 404 returned error can't find the container with id c270c29ff04945f8a9ee3a1930d2a15c434d0b05a98692b84b0e5117ffdffacd
Nov 01 03:39:36.622908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.622873    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:39:36.623025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.622938    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:39:36.623025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:36.622967    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:39:37.348352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:37.348310    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" event={"ID":"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1","Type":"ContainerStarted","Data":"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"}
Nov 01 03:39:37.350081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:37.350050    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" event={"ID":"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1","Type":"ContainerStarted","Data":"c270c29ff04945f8a9ee3a1930d2a15c434d0b05a98692b84b0e5117ffdffacd"}
Nov 01 03:39:37.607063 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:37.605732    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podStartSLOduration=2.605717862 podStartE2EDuration="2.605717862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:39:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:39:37.553633817 +0000 UTC m=+1943.969533344" watchObservedRunningTime="2026-11-01 03:39:37.605717862 +0000 UTC m=+1944.021617391"
Nov 01 03:39:38.173202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.173175    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:38.175413 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.175389    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 01 03:39:38.175495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.175458    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 01 03:39:38.398226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.398136    4892 generic.go:356] "Generic (PLEG): container finished" podID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerID="75100b94b63229f56653a0069c34d7f8c376c3a8d6707fdec1487f046e33a1ab" exitCode=0
Nov 01 03:39:38.398753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.398714    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" event={"ID":"175a95b4-59f0-4d78-9c11-10abc8ef3c2b","Type":"ContainerDied","Data":"75100b94b63229f56653a0069c34d7f8c376c3a8d6707fdec1487f046e33a1ab"}
Nov 01 03:39:38.451093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.451064    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:39:38.515993 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.515957    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\" (UniqueName: \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\") pod \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") "
Nov 01 03:39:38.516165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.516142    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\") pod \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\" (UID: \"175a95b4-59f0-4d78-9c11-10abc8ef3c2b\") "
Nov 01 03:39:38.516502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.516478    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume" pod "175a95b4-59f0-4d78-9c11-10abc8ef3c2b" (UID: "175a95b4-59f0-4d78-9c11-10abc8ef3c2b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:39:38.518013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.517959    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n" pod "175a95b4-59f0-4d78-9c11-10abc8ef3c2b" (UID: "175a95b4-59f0-4d78-9c11-10abc8ef3c2b"). InnerVolumeSpecName "kube-api-access-5456n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:39:38.618620 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.618579    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5456n\" (UniqueName: \"kubernetes.io/projected/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-kube-api-access-5456n\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:39:38.618804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:38.618631    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/175a95b4-59f0-4d78-9c11-10abc8ef3c2b-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:39:39.173306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.173267    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 01 03:39:39.173483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.173330    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 01 03:39:39.405846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.405813    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"
Nov 01 03:39:39.406348 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.405843    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk" event={"ID":"175a95b4-59f0-4d78-9c11-10abc8ef3c2b","Type":"ContainerDied","Data":"96fd83fe7ed15f7822d75983f316e83c80ddd4e81b5c29ba86891ebc05377ae3"}
Nov 01 03:39:39.406348 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.405902    4892 scope.go:122] "RemoveContainer" containerID="75100b94b63229f56653a0069c34d7f8c376c3a8d6707fdec1487f046e33a1ab"
Nov 01 03:39:39.573015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.572947    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:39:39.646026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.642327    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-fdbdfefd-6bcb9c8569-nzppk"]
Nov 01 03:39:39.702484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:39.702454    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" path="/var/lib/kubelet/pods/175a95b4-59f0-4d78-9c11-10abc8ef3c2b/volumes"
Nov 01 03:39:40.172951 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:40.172916    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 01 03:39:40.173159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:40.172999    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 01 03:39:41.173367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:41.173333    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 01 03:39:41.173872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:41.173396    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 01 03:39:42.173353 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:42.173315    4892 patch_prober.go:29] interesting pod/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused" start-of-body=
Nov 01 03:39:42.173537 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:42.173387    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.101:15021/healthz/ready\": dial tcp 10.134.0.101:15021: connect: connection refused"
Nov 01 03:39:43.234352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:43.234316    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:43.236351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:43.236324    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:43.239020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:39:43.238972    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:39:57.613034 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:39:57.612995    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:40:27.615337 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:40:27.615256    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:40:40.127877 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.127838    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:40:40.128314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.128078    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" podUID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" containerName="maas-api" containerID="cri-o://800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc" gracePeriod=30
Nov 01 03:40:40.479636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.479609    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:40:40.676671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.672504    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\" (UniqueName: \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\") pod \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") "
Nov 01 03:40:40.676671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.672586    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\") pod \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") "
Nov 01 03:40:40.676671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.672652    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") pod \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") "
Nov 01 03:40:40.676671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.672806    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\") pod \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\" (UID: \"dbb72c63-aa0d-4bc1-977f-96272a4560b3\") "
Nov 01 03:40:40.676671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.676248    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca" pod "dbb72c63-aa0d-4bc1-977f-96272a4560b3" (UID: "dbb72c63-aa0d-4bc1-977f-96272a4560b3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:40:40.690041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.690002    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs" pod "dbb72c63-aa0d-4bc1-977f-96272a4560b3" (UID: "dbb72c63-aa0d-4bc1-977f-96272a4560b3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:40:40.691371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.691341    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls" pod "dbb72c63-aa0d-4bc1-977f-96272a4560b3" (UID: "dbb72c63-aa0d-4bc1-977f-96272a4560b3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:40:40.693220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.693196    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd" pod "dbb72c63-aa0d-4bc1-977f-96272a4560b3" (UID: "dbb72c63-aa0d-4bc1-977f-96272a4560b3"). InnerVolumeSpecName "kube-api-access-9llsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:40:40.775903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.775660    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9llsd\" (UniqueName: \"kubernetes.io/projected/dbb72c63-aa0d-4bc1-977f-96272a4560b3-kube-api-access-9llsd\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:40:40.775903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.775701    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:40:40.775903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.775720    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dbb72c63-aa0d-4bc1-977f-96272a4560b3-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:40:40.775903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:40.775738    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbb72c63-aa0d-4bc1-977f-96272a4560b3-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:40:41.008935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.008903    4892 generic.go:356] "Generic (PLEG): container finished" podID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" containerID="800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc" exitCode=0
Nov 01 03:40:41.009180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.008948    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"
Nov 01 03:40:41.009180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.009023    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" event={"ID":"dbb72c63-aa0d-4bc1-977f-96272a4560b3","Type":"ContainerDied","Data":"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"}
Nov 01 03:40:41.009180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.009093    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb" event={"ID":"dbb72c63-aa0d-4bc1-977f-96272a4560b3","Type":"ContainerDied","Data":"92d3b0e78ebf82a1614cceefad3ddb9a1d01a18da5650e6c0439d894215c4d43"}
Nov 01 03:40:41.009180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.009123    4892 scope.go:122] "RemoveContainer" containerID="800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"
Nov 01 03:40:41.021321 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.021297    4892 scope.go:122] "RemoveContainer" containerID="800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"
Nov 01 03:40:41.021687 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:40:41.021665    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc\": container with ID starting with 800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc not found: ID does not exist" containerID="800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"
Nov 01 03:40:41.021800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.021696    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc"} err="failed to get container status \"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc\": rpc error: code = NotFound desc = could not find container \"800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc\": container with ID starting with 800fd4a2fa0a8f8b25b662504bbf993949cec27d341fe0690d6da067004107dc not found: ID does not exist"
Nov 01 03:40:41.204240 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.202599    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:40:41.207885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.207854    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-0cb10b-5545978955-psckb"]
Nov 01 03:40:41.701890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:40:41.701842    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" path="/var/lib/kubelet/pods/dbb72c63-aa0d-4bc1-977f-96272a4560b3/volumes"
Nov 01 03:40:57.615550 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:40:57.615511    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:41:06.927662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:06.927626    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:41:07.035259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:07.035160    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:41:11.928602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:11.928542    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" podUID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" containerName="payload-processing" containerID="cri-o://9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041" gracePeriod=25
Nov 01 03:41:12.035552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.035494    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" podUID="d4b4ccee-c768-4685-a695-6dd5dd551519" containerName="payload-pre-processing" containerID="cri-o://558cdc6cd28622618c201d66a92bc530005ecfa2f65d9e2725b7e5c47cc9e921" gracePeriod=25
Nov 01 03:41:12.236866 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.236832    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:41:12.248364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.248327    4892 generic.go:356] "Generic (PLEG): container finished" podID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" containerID="9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041" exitCode=0
Nov 01 03:41:12.248797 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.248763    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" event={"ID":"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b","Type":"ContainerDied","Data":"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"}
Nov 01 03:41:12.248901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.248865    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg" event={"ID":"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b","Type":"ContainerDied","Data":"40c461b5e10737a6dd5b513b870ad20f5619cf3e6a9d9b868badaf88a9f3949f"}
Nov 01 03:41:12.248901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.248895    4892 scope.go:122] "RemoveContainer" containerID="9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"
Nov 01 03:41:12.249050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.249030    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"
Nov 01 03:41:12.255154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.255130    4892 generic.go:356] "Generic (PLEG): container finished" podID="d4b4ccee-c768-4685-a695-6dd5dd551519" containerID="558cdc6cd28622618c201d66a92bc530005ecfa2f65d9e2725b7e5c47cc9e921" exitCode=0
Nov 01 03:41:12.255252 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.255185    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" event={"ID":"d4b4ccee-c768-4685-a695-6dd5dd551519","Type":"ContainerDied","Data":"558cdc6cd28622618c201d66a92bc530005ecfa2f65d9e2725b7e5c47cc9e921"}
Nov 01 03:41:12.265575 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.265536    4892 scope.go:122] "RemoveContainer" containerID="9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"
Nov 01 03:41:12.266660 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:41:12.266635    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041\": container with ID starting with 9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041 not found: ID does not exist" containerID="9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"
Nov 01 03:41:12.266826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.266797    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041"} err="failed to get container status \"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041\": rpc error: code = NotFound desc = could not find container \"9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041\": container with ID starting with 9660320c90d3149e9117c0894257b828a33a7f3767c71a63c4a2f65b13d1a041 not found: ID does not exist"
Nov 01 03:41:12.313223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.313179    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\" (UniqueName: \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\") pod \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") "
Nov 01 03:41:12.314291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.314254    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\") pod \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\" (UID: \"eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b\") "
Nov 01 03:41:12.314888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.314859    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume" pod "eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" (UID: "eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:41:12.315806 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.315778    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:12.316778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.316739    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v" pod "eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" (UID: "eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b"). InnerVolumeSpecName "kube-api-access-mlb5v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:41:12.404193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.404160    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:41:12.420040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.416419    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\" (UniqueName: \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\") pod \"d4b4ccee-c768-4685-a695-6dd5dd551519\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") "
Nov 01 03:41:12.420040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.416764    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\") pod \"d4b4ccee-c768-4685-a695-6dd5dd551519\" (UID: \"d4b4ccee-c768-4685-a695-6dd5dd551519\") "
Nov 01 03:41:12.420040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.417516    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mlb5v\" (UniqueName: \"kubernetes.io/projected/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b-kube-api-access-mlb5v\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:12.420040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.417590    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume" pod "d4b4ccee-c768-4685-a695-6dd5dd551519" (UID: "d4b4ccee-c768-4685-a695-6dd5dd551519"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:41:12.424848 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.424817    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw" pod "d4b4ccee-c768-4685-a695-6dd5dd551519" (UID: "d4b4ccee-c768-4685-a695-6dd5dd551519"). InnerVolumeSpecName "kube-api-access-t7zbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:41:12.518519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.518481    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4b4ccee-c768-4685-a695-6dd5dd551519-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:12.518519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.518520    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7zbw\" (UniqueName: \"kubernetes.io/projected/d4b4ccee-c768-4685-a695-6dd5dd551519-kube-api-access-t7zbw\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:12.675145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.675106    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:41:12.679798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:12.679764    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-634332-6c6775d85f-vxsbg"]
Nov 01 03:41:13.264142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.264111    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"
Nov 01 03:41:13.264142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.264134    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x" event={"ID":"d4b4ccee-c768-4685-a695-6dd5dd551519","Type":"ContainerDied","Data":"fc663dc6ecb118f4e9f9cd43b1edfda83cb0eb2c38f389e1a6d2308e43779876"}
Nov 01 03:41:13.264698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.264203    4892 scope.go:122] "RemoveContainer" containerID="558cdc6cd28622618c201d66a92bc530005ecfa2f65d9e2725b7e5c47cc9e921"
Nov 01 03:41:13.433301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.433266    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:41:13.481193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.481157    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-634332-f548c49b7-8lg6x"]
Nov 01 03:41:13.704114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.704031    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4b4ccee-c768-4685-a695-6dd5dd551519" path="/var/lib/kubelet/pods/d4b4ccee-c768-4685-a695-6dd5dd551519/volumes"
Nov 01 03:41:13.704568 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:13.704545    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" path="/var/lib/kubelet/pods/eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b/volumes"
Nov 01 03:41:21.945455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:21.945419    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:41:21.945915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:21.945660    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy" containerID="cri-o://427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f" gracePeriod=30
Nov 01 03:41:27.231048 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.231017    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:41:27.257325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257291    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257499 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257332    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257499 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257379    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257499 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257416    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257617    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257674    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257751 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257710    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257751 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257730    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:41:27.257857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257753    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257735    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:41:27.257974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257902    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\") pod \"663bb7b3-36a9-427e-8175-fe1b2978a3db\" (UID: \"663bb7b3-36a9-427e-8175-fe1b2978a3db\") "
Nov 01 03:41:27.257974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257895    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:41:27.258128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.257973    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:41:27.258190 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258173    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:41:27.258625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258598    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.258727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258629    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663bb7b3-36a9-427e-8175-fe1b2978a3db-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.258727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258646    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.258727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258660    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.258727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.258675    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.259831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.259801    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:41:27.259946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.259924    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:41:27.259946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.259942    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:41:27.260069 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.259932    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8" pod "663bb7b3-36a9-427e-8175-fe1b2978a3db" (UID: "663bb7b3-36a9-427e-8175-fe1b2978a3db"). InnerVolumeSpecName "kube-api-access-hqff8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:41:27.360549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.360509    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqff8\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-kube-api-access-hqff8\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.360549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.360544    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.360549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.360553    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.360802 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.360563    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663bb7b3-36a9-427e-8175-fe1b2978a3db-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:41:27.413642 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.413607    4892 generic.go:356] "Generic (PLEG): container finished" podID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerID="427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f" exitCode=0
Nov 01 03:41:27.413823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.413654    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"
Nov 01 03:41:27.413823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.413690    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerDied","Data":"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"}
Nov 01 03:41:27.413823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.413780    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz" event={"ID":"663bb7b3-36a9-427e-8175-fe1b2978a3db","Type":"ContainerDied","Data":"4a1e263e2c34d8223c1878bb2ae0cf8d5f22e1c6f888d329c4bb2f8ec9100b83"}
Nov 01 03:41:27.413823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.413809    4892 scope.go:122] "RemoveContainer" containerID="427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"
Nov 01 03:41:27.434546 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.434510    4892 scope.go:122] "RemoveContainer" containerID="056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"
Nov 01 03:41:27.445457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.445430    4892 scope.go:122] "RemoveContainer" containerID="427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"
Nov 01 03:41:27.445934 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:41:27.445899    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f\": container with ID starting with 427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f not found: ID does not exist" containerID="427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"
Nov 01 03:41:27.446065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.445944    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f"} err="failed to get container status \"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f\": rpc error: code = NotFound desc = could not find container \"427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f\": container with ID starting with 427f9c815a06725a0fa0579a37d1fd716d6f07a7653790709539a6d922428c0f not found: ID does not exist"
Nov 01 03:41:27.446065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.445969    4892 scope.go:122] "RemoveContainer" containerID="056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"
Nov 01 03:41:27.446366 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:41:27.446335    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1\": container with ID starting with 056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1 not found: ID does not exist" containerID="056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"
Nov 01 03:41:27.446453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.446375    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1"} err="failed to get container status \"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1\": rpc error: code = NotFound desc = could not find container \"056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1\": container with ID starting with 056dba2b4775f3f4d455c5f6ef82441d8c82ba63b1fc1bc4785322c0f8f6fdc1 not found: ID does not exist"
Nov 01 03:41:27.582514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.582477    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:41:27.586890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.586862    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-634332-openshift-default-7964cb5869-cgqvz"]
Nov 01 03:41:27.614843 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:41:27.614802    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:41:27.702362 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:27.702328    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" path="/var/lib/kubelet/pods/663bb7b3-36a9-427e-8175-fe1b2978a3db/volumes"
Nov 01 03:41:57.614799 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:41:57.614718    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:41:59.879370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:59.879325    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:41:59.991723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:41:59.991683    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:42:03.186968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:03.186931    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:42:03.187434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:03.187145    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" containerID="cri-o://f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f" gracePeriod=30
Nov 01 03:42:03.825883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:03.825844    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:42:03.826181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:03.826115    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" containerID="cri-o://3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8" gracePeriod=30
Nov 01 03:42:04.815226 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:04.815187    4892 patch_prober.go:29] interesting pod/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:42:04.815641 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:04.815243    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:42:04.879656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:04.879617    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" podUID="20623345-4e6b-4b43-9c95-a98412d8cf91" containerName="payload-processing" containerID="cri-o://1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204" gracePeriod=25
Nov 01 03:42:04.938080 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:04.938045    4892 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20623345_4e6b_4b43_9c95_a98412d8cf91.slice/crio-1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204.scope\": RecentStats: unable to find data in memory cache]"
Nov 01 03:42:04.992714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:04.992669    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" podUID="14627dce-7a61-4891-ab62-47b1082b875b" containerName="payload-pre-processing" containerID="cri-o://669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2" gracePeriod=25
Nov 01 03:42:05.165946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.165911    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:42:05.278372 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.278344    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:42:05.293018 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.292957    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\" (UniqueName: \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\") pod \"20623345-4e6b-4b43-9c95-a98412d8cf91\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") "
Nov 01 03:42:05.293200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.293123    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\") pod \"20623345-4e6b-4b43-9c95-a98412d8cf91\" (UID: \"20623345-4e6b-4b43-9c95-a98412d8cf91\") "
Nov 01 03:42:05.293741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.293711    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume" pod "20623345-4e6b-4b43-9c95-a98412d8cf91" (UID: "20623345-4e6b-4b43-9c95-a98412d8cf91"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:05.293948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.293924    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20623345-4e6b-4b43-9c95-a98412d8cf91-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:05.296244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.296212    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh" pod "20623345-4e6b-4b43-9c95-a98412d8cf91" (UID: "20623345-4e6b-4b43-9c95-a98412d8cf91"). InnerVolumeSpecName "kube-api-access-79mwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:05.326387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.326349    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:42:05.326557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.326419    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:42:05.394874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.394780    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\" (UniqueName: \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\") pod \"14627dce-7a61-4891-ab62-47b1082b875b\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") "
Nov 01 03:42:05.395102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.395069    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\") pod \"14627dce-7a61-4891-ab62-47b1082b875b\" (UID: \"14627dce-7a61-4891-ab62-47b1082b875b\") "
Nov 01 03:42:05.395415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.395390    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume" pod "14627dce-7a61-4891-ab62-47b1082b875b" (UID: "14627dce-7a61-4891-ab62-47b1082b875b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:05.395602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.395587    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/14627dce-7a61-4891-ab62-47b1082b875b-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:05.395647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.395605    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79mwh\" (UniqueName: \"kubernetes.io/projected/20623345-4e6b-4b43-9c95-a98412d8cf91-kube-api-access-79mwh\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:05.397035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.397011    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b" pod "14627dce-7a61-4891-ab62-47b1082b875b" (UID: "14627dce-7a61-4891-ab62-47b1082b875b"). InnerVolumeSpecName "kube-api-access-77l5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:05.496711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.496673    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77l5b\" (UniqueName: \"kubernetes.io/projected/14627dce-7a61-4891-ab62-47b1082b875b-kube-api-access-77l5b\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:05.790312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.790280    4892 generic.go:356] "Generic (PLEG): container finished" podID="14627dce-7a61-4891-ab62-47b1082b875b" containerID="669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2" exitCode=0
Nov 01 03:42:05.790487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.790333    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"
Nov 01 03:42:05.790487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.790359    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" event={"ID":"14627dce-7a61-4891-ab62-47b1082b875b","Type":"ContainerDied","Data":"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"}
Nov 01 03:42:05.790487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.790414    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk" event={"ID":"14627dce-7a61-4891-ab62-47b1082b875b","Type":"ContainerDied","Data":"5f5d22fc950d26dc347ff8058070f871f555d6f8e1b5358e9d679e16230bf437"}
Nov 01 03:42:05.790487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.790439    4892 scope.go:122] "RemoveContainer" containerID="669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"
Nov 01 03:42:05.791959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.791921    4892 generic.go:356] "Generic (PLEG): container finished" podID="20623345-4e6b-4b43-9c95-a98412d8cf91" containerID="1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204" exitCode=0
Nov 01 03:42:05.792107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.791989    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"
Nov 01 03:42:05.792164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.791990    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" event={"ID":"20623345-4e6b-4b43-9c95-a98412d8cf91","Type":"ContainerDied","Data":"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"}
Nov 01 03:42:05.792223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.792193    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq" event={"ID":"20623345-4e6b-4b43-9c95-a98412d8cf91","Type":"ContainerDied","Data":"e86ff668b12329f6134fdfa989751f918ef361c5ef3694d1811ea519e9bb98bc"}
Nov 01 03:42:05.801872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.801412    4892 scope.go:122] "RemoveContainer" containerID="669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"
Nov 01 03:42:05.801872 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:05.801708    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2\": container with ID starting with 669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2 not found: ID does not exist" containerID="669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"
Nov 01 03:42:05.801872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.801732    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2"} err="failed to get container status \"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2\": rpc error: code = NotFound desc = could not find container \"669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2\": container with ID starting with 669c4a2c1b1f343fc5573e56fbfc969a5fef1f56b880024c0268b34ae4afb3e2 not found: ID does not exist"
Nov 01 03:42:05.801872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.801749    4892 scope.go:122] "RemoveContainer" containerID="1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"
Nov 01 03:42:05.811144 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.811124    4892 scope.go:122] "RemoveContainer" containerID="1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"
Nov 01 03:42:05.811424 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:05.811405    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204\": container with ID starting with 1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204 not found: ID does not exist" containerID="1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"
Nov 01 03:42:05.811478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.811431    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204"} err="failed to get container status \"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204\": rpc error: code = NotFound desc = could not find container \"1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204\": container with ID starting with 1d63f74975d68b5a1a1888dda0aaba6c4271e9845e4c5971d1ae732c54ce3204 not found: ID does not exist"
Nov 01 03:42:05.960093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.960058    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:42:05.968077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:05.967994    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w4-2058fb-6c67569ff9-njxtk"]
Nov 01 03:42:06.080764 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:06.080732    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:42:06.083661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:06.083638    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w4-2058fb-84dd4c4847-fcdbq"]
Nov 01 03:42:07.702506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:07.702471    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14627dce-7a61-4891-ab62-47b1082b875b" path="/var/lib/kubelet/pods/14627dce-7a61-4891-ab62-47b1082b875b/volumes"
Nov 01 03:42:07.702971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:07.702955    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20623345-4e6b-4b43-9c95-a98412d8cf91" path="/var/lib/kubelet/pods/20623345-4e6b-4b43-9c95-a98412d8cf91/volumes"
Nov 01 03:42:08.470711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.470638    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:42:08.636767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.636732    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.636937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.636847    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.636937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.636881    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.636963    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637049    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637198 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637179    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637229    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:08.637281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637258    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:08.637393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637278    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637306    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637334    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\") pod \"2f03dc66-e6b6-44e0-813b-364d5061ff33\" (UID: \"2f03dc66-e6b6-44e0-813b-364d5061ff33\") "
Nov 01 03:42:08.637393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637365    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:08.637631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637565    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:08.637688 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.637635    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:08.638082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.638052    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.638082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.638081    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.638291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.638096    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.638291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.638111    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f03dc66-e6b6-44e0-813b-364d5061ff33-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.638291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.638125    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.639304 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.639268    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:08.639488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.639459    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "kube-api-access-z5vhv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:08.639588 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.639490    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:08.639633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.639583    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token" pod "2f03dc66-e6b6-44e0-813b-364d5061ff33" (UID: "2f03dc66-e6b6-44e0-813b-364d5061ff33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:08.739850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.739753    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.739850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.739787    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.739850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.739800    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5vhv\" (UniqueName: \"kubernetes.io/projected/2f03dc66-e6b6-44e0-813b-364d5061ff33-kube-api-access-z5vhv\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.739850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.739815    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f03dc66-e6b6-44e0-813b-364d5061ff33-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:08.814602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.814565    4892 generic.go:356] "Generic (PLEG): container finished" podID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerID="f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f" exitCode=0
Nov 01 03:42:08.814780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.814610    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"
Nov 01 03:42:08.814780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.814648    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" event={"ID":"2f03dc66-e6b6-44e0-813b-364d5061ff33","Type":"ContainerDied","Data":"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"}
Nov 01 03:42:08.814780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.814709    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q" event={"ID":"2f03dc66-e6b6-44e0-813b-364d5061ff33","Type":"ContainerDied","Data":"29d8a17f835bbc1712fb0ddeb313e6bf5f8ffc9ed761f3d9c748423fb6d24fd4"}
Nov 01 03:42:08.814780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.814734    4892 scope.go:122] "RemoveContainer" containerID="f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"
Nov 01 03:42:08.825487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.825468    4892 scope.go:122] "RemoveContainer" containerID="f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"
Nov 01 03:42:08.825789 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:08.825770    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f\": container with ID starting with f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f not found: ID does not exist" containerID="f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"
Nov 01 03:42:08.825838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.825799    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f"} err="failed to get container status \"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f\": rpc error: code = NotFound desc = could not find container \"f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f\": container with ID starting with f040284d25122f6e4287bd1e61940e5ff50f7229fa260dae72dcc8284a745b2f not found: ID does not exist"
Nov 01 03:42:08.981937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.981902    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:42:08.986403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:08.986373    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-r2k7q"]
Nov 01 03:42:09.089208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.089175    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:42:09.247632 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.247588    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.247828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.247764    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.247896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.247885    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.247971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.247952    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248054    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248292 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248272    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248380 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248205    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:09.248380 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248341    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248405    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248435    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\") pod \"965808b9-89ff-4911-837d-bd3f9b2b4527\" (UID: \"965808b9-89ff-4911-837d-bd3f9b2b4527\") "
Nov 01 03:42:09.248610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248526    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:09.248662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248617    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:09.248725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.248663    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:09.249173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.249146    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.249293 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.249178    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.249293 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.249194    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.249293 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.249207    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.249501 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.249477    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:09.250909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.250880    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "kube-api-access-9dlhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:09.255542 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.255467    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:09.255680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.255636    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:09.255680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.255650    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token" pod "965808b9-89ff-4911-837d-bd3f9b2b4527" (UID: "965808b9-89ff-4911-837d-bd3f9b2b4527"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:09.350692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.350648    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.350692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.350681    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/965808b9-89ff-4911-837d-bd3f9b2b4527-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.350692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.350696    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.350944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.350707    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/965808b9-89ff-4911-837d-bd3f9b2b4527-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.350944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.350720    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dlhb\" (UniqueName: \"kubernetes.io/projected/965808b9-89ff-4911-837d-bd3f9b2b4527-kube-api-access-9dlhb\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:09.702391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.702314    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" path="/var/lib/kubelet/pods/2f03dc66-e6b6-44e0-813b-364d5061ff33/volumes"
Nov 01 03:42:09.823755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.823724    4892 generic.go:356] "Generic (PLEG): container finished" podID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerID="3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8" exitCode=0
Nov 01 03:42:09.824244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.823771    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"
Nov 01 03:42:09.824244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.823796    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" event={"ID":"965808b9-89ff-4911-837d-bd3f9b2b4527","Type":"ContainerDied","Data":"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"}
Nov 01 03:42:09.824244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.823849    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb" event={"ID":"965808b9-89ff-4911-837d-bd3f9b2b4527","Type":"ContainerDied","Data":"a4ed5c87cbb0c8241fef76ed83dd04d97c90fad06e04349a0874e1020ce1ad8f"}
Nov 01 03:42:09.824244 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.823870    4892 scope.go:122] "RemoveContainer" containerID="3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"
Nov 01 03:42:09.833727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.833706    4892 scope.go:122] "RemoveContainer" containerID="3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"
Nov 01 03:42:09.834010 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:09.833989    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8\": container with ID starting with 3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8 not found: ID does not exist" containerID="3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"
Nov 01 03:42:09.834065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.834020    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8"} err="failed to get container status \"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8\": rpc error: code = NotFound desc = could not find container \"3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8\": container with ID starting with 3898bac4e0b3e45c630dad4f2f8d7548fda81415fb760c6f7ce5091a04fbb1e8 not found: ID does not exist"
Nov 01 03:42:09.992315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:09.992231    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:42:10.042124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:10.042092    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-vvwzb"]
Nov 01 03:42:11.208817 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.208782    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:42:11.209292 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.209024    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy" containerID="cri-o://4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c" gracePeriod=30
Nov 01 03:42:11.259243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.259201    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:42:11.259455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.259430    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy" containerID="cri-o://ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376" gracePeriod=30
Nov 01 03:42:11.262020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.261968    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:42:11.262188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.262163    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" containerID="cri-o://343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4" gracePeriod=30
Nov 01 03:42:11.308857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.308826    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:42:11.309078 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.309024    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy" containerID="cri-o://2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c" gracePeriod=30
Nov 01 03:42:11.701790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:11.701754    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" path="/var/lib/kubelet/pods/965808b9-89ff-4911-837d-bd3f9b2b4527/volumes"
Nov 01 03:42:13.775619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:13.775580    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:42:13.776059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:13.775647    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:42:13.966285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:13.966257    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:42:13.971325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:13.971292    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:42:16.574907 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.574879    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:42:16.619771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.619306    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:42:16.670459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670435    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.670578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670475    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.670578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670527    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.670578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670572    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.670752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670598    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.670752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670648    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.670752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670672    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.670752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670694    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.670752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670719    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.671017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670734    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.671017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670742    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.671017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670820    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.671017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670903    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.670949    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671032    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671071    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\") pod \"e04085b7-baf7-445d-93e2-491f6810ccba\" (UID: \"e04085b7-baf7-445d-93e2-491f6810ccba\") "
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671106    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671112    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671150    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671206    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671219    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:16.671284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671261    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671773 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671330    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\") pod \"d59e7721-5189-4c9c-930c-c2d35258a032\" (UID: \"d59e7721-5189-4c9c-930c-c2d35258a032\") "
Nov 01 03:42:16.671773 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671437    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.671920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.671895    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.672497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672468    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.672601 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672505    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.672931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672909    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672940    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e04085b7-baf7-445d-93e2-491f6810ccba-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672957    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672966    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.672974    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.673025    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.673035    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.673044    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.673052    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.673483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.673454    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:16.674166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674140    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:16.674567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674542    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.674663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674582    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:16.674663 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674646    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.674779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674668    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "kube-api-access-pl2gg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.674779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.674678    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token" pod "e04085b7-baf7-445d-93e2-491f6810ccba" (UID: "e04085b7-baf7-445d-93e2-491f6810ccba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.675743 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.675723    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "kube-api-access-s97wv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.676257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.676239    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy" pod "d59e7721-5189-4c9c-930c-c2d35258a032" (UID: "d59e7721-5189-4c9c-930c-c2d35258a032"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.679075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.679011    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:42:16.681219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.681199    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:42:16.773856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773775    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.773856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773819    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.773856 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773842    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773871    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773898    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.773939    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774010    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774066    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774098    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774139    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774138    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774137    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774222    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774259    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774340    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774402    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774507    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774528    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.774574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774574    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.775005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774625    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\") pod \"f2a19921-1271-436a-ab90-26a79ff9e83c\" (UID: \"f2a19921-1271-436a-ab90-26a79ff9e83c\") "
Nov 01 03:42:16.775005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774659    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\") pod \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\" (UID: \"90ec406b-0bb7-426e-8e3a-c7596539bb8c\") "
Nov 01 03:42:16.775005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774743    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.775005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774749    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.775005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.774785    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:16.775294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775104    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.775447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775425    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e04085b7-baf7-445d-93e2-491f6810ccba-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775452    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775469    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775486    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d59e7721-5189-4c9c-930c-c2d35258a032-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775532    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e04085b7-baf7-445d-93e2-491f6810ccba-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775547    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775561    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d59e7721-5189-4c9c-930c-c2d35258a032-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775562    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775576    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d59e7721-5189-4c9c-930c-c2d35258a032-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775591    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775608    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.775625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775622    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl2gg\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-kube-api-access-pl2gg\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775636    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775649    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775663    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s97wv\" (UniqueName: \"kubernetes.io/projected/d59e7721-5189-4c9c-930c-c2d35258a032-kube-api-access-s97wv\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775678    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775691    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e04085b7-baf7-445d-93e2-491f6810ccba-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775903    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.776117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.775937    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:16.776891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.776861    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:16.777329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.777306    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.777461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.777441    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "kube-api-access-wr856". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.778296 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.778273    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token" pod "90ec406b-0bb7-426e-8e3a-c7596539bb8c" (UID: "90ec406b-0bb7-426e-8e3a-c7596539bb8c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.778402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.778344    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:16.778619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.778594    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:16.778679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.778619    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.779363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.779345    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48" pod "f2a19921-1271-436a-ab90-26a79ff9e83c" (UID: "f2a19921-1271-436a-ab90-26a79ff9e83c"). InnerVolumeSpecName "kube-api-access-t5d48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:16.873518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.873485    4892 generic.go:356] "Generic (PLEG): container finished" podID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerID="2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c" exitCode=0
Nov 01 03:42:16.873677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.873563    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" event={"ID":"f2a19921-1271-436a-ab90-26a79ff9e83c","Type":"ContainerDied","Data":"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"}
Nov 01 03:42:16.873677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.873601    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r" event={"ID":"f2a19921-1271-436a-ab90-26a79ff9e83c","Type":"ContainerDied","Data":"34cec4185c12dd8a4626bc11223d98ae66b71b460687460781f8ce25cbcb60ec"}
Nov 01 03:42:16.873677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.873625    4892 scope.go:122] "RemoveContainer" containerID="2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"
Nov 01 03:42:16.873825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.873735    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"
Nov 01 03:42:16.881075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881038    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881075 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881074    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881090    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/90ec406b-0bb7-426e-8e3a-c7596539bb8c-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881108    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f2a19921-1271-436a-ab90-26a79ff9e83c-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881121    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881136    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr856\" (UniqueName: \"kubernetes.io/projected/90ec406b-0bb7-426e-8e3a-c7596539bb8c-kube-api-access-wr856\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881151    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881164    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/90ec406b-0bb7-426e-8e3a-c7596539bb8c-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881177    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881191    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5d48\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-kube-api-access-t5d48\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.881301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.881205    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f2a19921-1271-436a-ab90-26a79ff9e83c-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.884762    4892 generic.go:356] "Generic (PLEG): container finished" podID="e04085b7-baf7-445d-93e2-491f6810ccba" containerID="4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c" exitCode=0
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.884960    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" event={"ID":"e04085b7-baf7-445d-93e2-491f6810ccba","Type":"ContainerDied","Data":"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"}
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.885044    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp" event={"ID":"e04085b7-baf7-445d-93e2-491f6810ccba","Type":"ContainerDied","Data":"6c13ab80fc06597f5ecaa33061e4e67793a65d2d72b1b110d718ec437d18b0ba"}
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.884974    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.887241    4892 generic.go:356] "Generic (PLEG): container finished" podID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerID="ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376" exitCode=0
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.887300    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" event={"ID":"90ec406b-0bb7-426e-8e3a-c7596539bb8c","Type":"ContainerDied","Data":"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"}
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.887331    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l" event={"ID":"90ec406b-0bb7-426e-8e3a-c7596539bb8c","Type":"ContainerDied","Data":"32bf95b87c9b76db461c5145882c9360b7273bb95a99c5528e2faf7cd845856d"}
Nov 01 03:42:16.890040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.887376    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"
Nov 01 03:42:16.893211 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.893182    4892 generic.go:356] "Generic (PLEG): container finished" podID="d59e7721-5189-4c9c-930c-c2d35258a032" containerID="343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4" exitCode=0
Nov 01 03:42:16.893328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.893253    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" event={"ID":"d59e7721-5189-4c9c-930c-c2d35258a032","Type":"ContainerDied","Data":"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"}
Nov 01 03:42:16.893328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.893295    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd" event={"ID":"d59e7721-5189-4c9c-930c-c2d35258a032","Type":"ContainerDied","Data":"db2a60ed6c5f9573152a0db3d689bc9bf5dad9051abfaaba979b6c5e45222cc2"}
Nov 01 03:42:16.893455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.893362    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"
Nov 01 03:42:16.896601 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.896574    4892 scope.go:122] "RemoveContainer" containerID="2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"
Nov 01 03:42:16.898329 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:16.898138    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c\": container with ID starting with 2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c not found: ID does not exist" containerID="2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"
Nov 01 03:42:16.898329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.898197    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c"} err="failed to get container status \"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c\": rpc error: code = NotFound desc = could not find container \"2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c\": container with ID starting with 2ca208f2b31cda90158fe13337b176e8005022b600fcd7f44e53f8d20cb5313c not found: ID does not exist"
Nov 01 03:42:16.898329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.898245    4892 scope.go:122] "RemoveContainer" containerID="4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"
Nov 01 03:42:16.911385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.911358    4892 scope.go:122] "RemoveContainer" containerID="4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"
Nov 01 03:42:16.911743 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:16.911724    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c\": container with ID starting with 4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c not found: ID does not exist" containerID="4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"
Nov 01 03:42:16.911837 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.911757    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c"} err="failed to get container status \"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c\": rpc error: code = NotFound desc = could not find container \"4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c\": container with ID starting with 4dd571fa0499a1f58458ec6d162dc4329a1520cdb5a9308e7e14c5513eb7729c not found: ID does not exist"
Nov 01 03:42:16.911837 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.911782    4892 scope.go:122] "RemoveContainer" containerID="ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"
Nov 01 03:42:16.922806 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.922784    4892 scope.go:122] "RemoveContainer" containerID="ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"
Nov 01 03:42:16.923119 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:16.923097    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376\": container with ID starting with ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376 not found: ID does not exist" containerID="ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"
Nov 01 03:42:16.923210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.923129    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376"} err="failed to get container status \"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376\": rpc error: code = NotFound desc = could not find container \"ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376\": container with ID starting with ed333bfd6312e419f0a20a1e0ab2aaca70fd1133758f041fa12c90cf3aa3b376 not found: ID does not exist"
Nov 01 03:42:16.923210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.923154    4892 scope.go:122] "RemoveContainer" containerID="343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"
Nov 01 03:42:16.931814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.931791    4892 scope.go:122] "RemoveContainer" containerID="343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"
Nov 01 03:42:16.932109 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:16.932092    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4\": container with ID starting with 343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4 not found: ID does not exist" containerID="343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"
Nov 01 03:42:16.932170 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:16.932115    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4"} err="failed to get container status \"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4\": rpc error: code = NotFound desc = could not find container \"343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4\": container with ID starting with 343a382943d41f0aac77b29deb1214ba1ccc1a70f88faded6461d8c7f31fddc4 not found: ID does not exist"
Nov 01 03:42:17.234351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.234271    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:42:17.237000 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.236965    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-pfv7l"]
Nov 01 03:42:17.348387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.348334    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:42:17.353061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.353033    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-hfnvd"]
Nov 01 03:42:17.465960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.465924    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:42:17.486465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.486354    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-j2sgp"]
Nov 01 03:42:17.616899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.616861    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:42:17.648341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.648284    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-2w72r"]
Nov 01 03:42:17.705141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.705106    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" path="/var/lib/kubelet/pods/90ec406b-0bb7-426e-8e3a-c7596539bb8c/volumes"
Nov 01 03:42:17.705696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.705680    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" path="/var/lib/kubelet/pods/d59e7721-5189-4c9c-930c-c2d35258a032/volumes"
Nov 01 03:42:17.706207 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.706189    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" path="/var/lib/kubelet/pods/e04085b7-baf7-445d-93e2-491f6810ccba/volumes"
Nov 01 03:42:17.706998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:17.706956    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" path="/var/lib/kubelet/pods/f2a19921-1271-436a-ab90-26a79ff9e83c/volumes"
Nov 01 03:42:26.642458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:26.642424    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:42:26.642948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:26.642651    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" containerID="cri-o://d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63" gracePeriod=30
Nov 01 03:42:26.669071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:26.669038    4892 patch_prober.go:29] interesting pod/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:42:26.669242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:26.669109    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:42:27.611992 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:27.611939    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:42:29.026699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:29.026668    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:42:29.027164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:29.026879    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy" containerID="cri-o://27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca" gracePeriod=30
Nov 01 03:42:29.045511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:29.045474    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:42:29.045750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:29.045725    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy" containerID="cri-o://a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc" gracePeriod=30
Nov 01 03:42:30.388835 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.388795    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:42:30.389351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389331    4892 cpu_manager.go:414] "containerMap: removing container" podUID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" containerName="maas-api"
Nov 01 03:42:30.389351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389347    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" containerName="maas-api"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389364    4892 cpu_manager.go:414] "containerMap: removing container" podUID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" containerName="payload-processing"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389370    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" containerName="payload-processing"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389378    4892 cpu_manager.go:414] "containerMap: removing container" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389383    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389397    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389402    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389413    4892 cpu_manager.go:414] "containerMap: removing container" podUID="14627dce-7a61-4891-ab62-47b1082b875b" containerName="payload-pre-processing"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389418    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14627dce-7a61-4891-ab62-47b1082b875b" containerName="payload-pre-processing"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389426    4892 cpu_manager.go:414] "containerMap: removing container" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy"
Nov 01 03:42:30.389429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389431    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389438    4892 cpu_manager.go:414] "containerMap: removing container" podUID="20623345-4e6b-4b43-9c95-a98412d8cf91" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389443    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20623345-4e6b-4b43-9c95-a98412d8cf91" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389452    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389457    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389466    4892 cpu_manager.go:414] "containerMap: removing container" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389471    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389479    4892 cpu_manager.go:414] "containerMap: removing container" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389483    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389495    4892 cpu_manager.go:414] "containerMap: removing container" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389500    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389507    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389511    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389519    4892 cpu_manager.go:414] "containerMap: removing container" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389523    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389531    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d4b4ccee-c768-4685-a695-6dd5dd551519" containerName="payload-pre-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389535    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4b4ccee-c768-4685-a695-6dd5dd551519" containerName="payload-pre-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389609    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d59e7721-5189-4c9c-930c-c2d35258a032" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389618    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389624    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="175a95b4-59f0-4d78-9c11-10abc8ef3c2b" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389630    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="965808b9-89ff-4911-837d-bd3f9b2b4527" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389637    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e04085b7-baf7-445d-93e2-491f6810ccba" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389644    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="663bb7b3-36a9-427e-8175-fe1b2978a3db" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389650    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4b4ccee-c768-4685-a695-6dd5dd551519" containerName="payload-pre-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389656    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbb72c63-aa0d-4bc1-977f-96272a4560b3" containerName="maas-api"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389663    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="eea8e1eb-fbc4-4cb9-ae94-b090ddebe92b" containerName="payload-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389669    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f03dc66-e6b6-44e0-813b-364d5061ff33" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389676    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f2a19921-1271-436a-ab90-26a79ff9e83c" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389683    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="14627dce-7a61-4891-ab62-47b1082b875b" containerName="payload-pre-processing"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389689    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="90ec406b-0bb7-426e-8e3a-c7596539bb8c" containerName="istio-proxy"
Nov 01 03:42:30.389807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.389694    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="20623345-4e6b-4b43-9c95-a98412d8cf91" containerName="payload-processing"
Nov 01 03:42:30.395138 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.395105    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.502596 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.499313    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5fd6cf-openshift-default-dockercfg-ggkvm\""
Nov 01 03:42:30.612863 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.612819    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.612926    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613018    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613085    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613159    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613227    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613260    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66st8\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613299    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.613493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.613331    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.626336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.626302    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:42:30.715708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715608    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.715708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715673    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.715958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715791    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66st8\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.715958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715831    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.715958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715858    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.715972    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716140 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716061    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716131    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716144    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716226    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716400 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716256    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716403    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716451    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.716925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.716903    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.718335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.718310    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.718480 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.718455    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.789618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.789577    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:30.789776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:30.789711    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66st8\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\") pod \"e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") " pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:31.022326 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.022285    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:31.460655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.460623    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:42:31.462420 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:31.462394    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c2924a3_74cb_46ca_9c69_3a9b8df0efc8.slice/crio-7374d0fe69e165947fbec60400cd25d26b1eef53797873cc3426999525f6a12b WatchSource:0}: Error finding container 7374d0fe69e165947fbec60400cd25d26b1eef53797873cc3426999525f6a12b: Status 404 returned error can't find the container with id 7374d0fe69e165947fbec60400cd25d26b1eef53797873cc3426999525f6a12b
Nov 01 03:42:31.465243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.465206    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:31.465336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.465288    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:31.465394 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.465337    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:31.939892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:31.939856    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:42:32.012393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.012360    4892 generic.go:356] "Generic (PLEG): container finished" podID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerID="d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63" exitCode=0
Nov 01 03:42:32.012606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.012421    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"
Nov 01 03:42:32.012606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.012446    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerDied","Data":"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"}
Nov 01 03:42:32.012606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.012508    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k" event={"ID":"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97","Type":"ContainerDied","Data":"6b4ea4bfda77351b520c7f200c61df13a778a28ff1b8f9834f9bb7b067fdb0a2"}
Nov 01 03:42:32.012606 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.012538    4892 scope.go:122] "RemoveContainer" containerID="d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"
Nov 01 03:42:32.014432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.014408    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerStarted","Data":"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"}
Nov 01 03:42:32.014560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.014454    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerStarted","Data":"7374d0fe69e165947fbec60400cd25d26b1eef53797873cc3426999525f6a12b"}
Nov 01 03:42:32.023152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.023032    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:32.023152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.023098    4892 scope.go:122] "RemoveContainer" containerID="0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"
Nov 01 03:42:32.030956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.030929    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:32.033156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.033035    4892 scope.go:122] "RemoveContainer" containerID="d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"
Nov 01 03:42:32.033408 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:32.033386    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63\": container with ID starting with d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63 not found: ID does not exist" containerID="d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"
Nov 01 03:42:32.033481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.033417    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63"} err="failed to get container status \"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63\": rpc error: code = NotFound desc = could not find container \"d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63\": container with ID starting with d66090015a358a2281fad782cccf1cfcd5f1899fba6a50c97a74f413be065a63 not found: ID does not exist"
Nov 01 03:42:32.033481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.033436    4892 scope.go:122] "RemoveContainer" containerID="0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"
Nov 01 03:42:32.033703 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:32.033683    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83\": container with ID starting with 0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83 not found: ID does not exist" containerID="0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"
Nov 01 03:42:32.033753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.033708    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83"} err="failed to get container status \"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83\": rpc error: code = NotFound desc = could not find container \"0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83\": container with ID starting with 0e524b4568d0b85e3d06f79e60140bb0a59781f98e1e7445318df738bff2cf83 not found: ID does not exist"
Nov 01 03:42:32.036160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036140    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036197    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036352    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036392    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036395    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:32.036459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036449    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036492    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036522    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036619 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036589    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036623    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\") pod \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\" (UID: \"fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97\") "
Nov 01 03:42:32.036822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.036770    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:32.037047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037020    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:32.037188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037168    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.037256 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037194    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.037256 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037209    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.037367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037268    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:32.037487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.037463    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:32.038958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.038933    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "kube-api-access-9l5lt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:32.039179 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.039158    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:32.039330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.039302    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:32.039506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.039489    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy" pod "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" (UID: "fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:32.138364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138323    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.138364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138363    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l5lt\" (UniqueName: \"kubernetes.io/projected/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-kube-api-access-9l5lt\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.138594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138381    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.138594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138396    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.138594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138410    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.138594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.138434    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:32.240202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.240143    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podStartSLOduration=2.240127941 podStartE2EDuration="2.240127941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:32.236955993 +0000 UTC m=+2118.652855547" watchObservedRunningTime="2026-11-01 03:42:32.240127941 +0000 UTC m=+2118.656027470"
Nov 01 03:42:32.441680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.441640    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:42:32.447096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:32.447060    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-2058fb-openshift-default-759c68489c-h992k"]
Nov 01 03:42:33.022370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:33.022340    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:33.023430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:33.023408    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:42:33.701776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:33.701743    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" path="/var/lib/kubelet/pods/fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97/volumes"
Nov 01 03:42:33.808615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:33.808580    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:42:33.808822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:33.808782    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery" containerID="cri-o://b9f73d025ff9648df9d15fa754ebfe0956affe3d28b3245ecf1086c4611aacaa" gracePeriod=30
Nov 01 03:42:34.033695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.033658    4892 generic.go:356] "Generic (PLEG): container finished" podID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerID="b9f73d025ff9648df9d15fa754ebfe0956affe3d28b3245ecf1086c4611aacaa" exitCode=0
Nov 01 03:42:34.034198 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.033746    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" event={"ID":"e08c0f6c-80eb-49ea-b362-07767a84306c","Type":"ContainerDied","Data":"b9f73d025ff9648df9d15fa754ebfe0956affe3d28b3245ecf1086c4611aacaa"}
Nov 01 03:42:34.199703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.199669    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:42:34.266163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266130    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.266332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266253    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.266332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266288    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.266332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266322    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.267464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266540    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.267464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266602    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.267464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266796    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:34.267464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.266815    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\") pod \"e08c0f6c-80eb-49ea-b362-07767a84306c\" (UID: \"e08c0f6c-80eb-49ea-b362-07767a84306c\") "
Nov 01 03:42:34.267464 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.267430    4892 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-ca-configmap\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.269783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.269749    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:42:34.270905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.270877    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.271052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.270903    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.271052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.270918    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:42:34.271052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.270971    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "kube-api-access-xbsf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.271858 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.271837    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert" pod "e08c0f6c-80eb-49ea-b362-07767a84306c" (UID: "e08c0f6c-80eb-49ea-b362-07767a84306c"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:42:34.312789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.312750    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:42:34.363685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.363656    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:42:34.368855 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.368816    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.368883    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.368912    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.368956    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369060    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369157    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369163    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.369333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369198    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.369387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369336    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369378    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.369453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369388    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369459    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\") pod \"015c1cd8-f0e0-4ccf-8719-51b08452764e\" (UID: \"015c1cd8-f0e0-4ccf-8719-51b08452764e\") "
Nov 01 03:42:34.369612 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369588    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.369757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.369730    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:34.370134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370108    4892 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e08c0f6c-80eb-49ea-b362-07767a84306c-local-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370137    4892 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-kubeconfig\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370153    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xbsf2\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-kube-api-access-xbsf2\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370166    4892 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-cacerts\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370182    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370195    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370209    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370230    4892 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-csr-dns-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370245    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e08c0f6c-80eb-49ea-b362-07767a84306c-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370257    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/015c1cd8-f0e0-4ccf-8719-51b08452764e-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.370745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.370270    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.371903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.371877    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.372016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.371926    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.372016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.371947    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "kube-api-access-pflbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.372016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.371968    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo" pod "015c1cd8-f0e0-4ccf-8719-51b08452764e" (UID: "015c1cd8-f0e0-4ccf-8719-51b08452764e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:34.471239 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471202    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471284    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471384    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471487    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471550 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471531    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471585    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471664 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471613    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471674    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471689    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.471771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471755    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\") pod \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\" (UID: \"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1\") "
Nov 01 03:42:34.471878 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471823    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.471930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.471888    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:34.472238 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472063    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.472346 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472319    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472342    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.472416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472352    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472388    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472405    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472423    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pflbt\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-kube-api-access-pflbt\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472439    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472453    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/015c1cd8-f0e0-4ccf-8719-51b08452764e-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.472624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.472467    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.474017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.473972    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:42:34.474017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.474009    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:42:34.474567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.474543    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.474829 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.474809    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58" pod "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" (UID: "61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1"). InnerVolumeSpecName "kube-api-access-jkk58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:34.574815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.574730    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.574815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.574775    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkk58\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-kube-api-access-jkk58\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.574815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.574793    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.574815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.574806    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:34.575119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:34.574822    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:35.041822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.041790    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"
Nov 01 03:42:35.041822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.041798    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv" event={"ID":"e08c0f6c-80eb-49ea-b362-07767a84306c","Type":"ContainerDied","Data":"cd6e94be8f5943325ff375f6f49ced926df268f3dc30180771cd2c7f6de09226"}
Nov 01 03:42:35.042448 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.041864    4892 scope.go:122] "RemoveContainer" containerID="b9f73d025ff9648df9d15fa754ebfe0956affe3d28b3245ecf1086c4611aacaa"
Nov 01 03:42:35.043590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.043495    4892 generic.go:356] "Generic (PLEG): container finished" podID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerID="a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc" exitCode=0
Nov 01 03:42:35.043590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.043532    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" event={"ID":"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1","Type":"ContainerDied","Data":"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"}
Nov 01 03:42:35.043590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.043571    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"
Nov 01 03:42:35.043769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.043593    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7" event={"ID":"61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1","Type":"ContainerDied","Data":"c270c29ff04945f8a9ee3a1930d2a15c434d0b05a98692b84b0e5117ffdffacd"}
Nov 01 03:42:35.045059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.045035    4892 generic.go:356] "Generic (PLEG): container finished" podID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerID="27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca" exitCode=0
Nov 01 03:42:35.045162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.045086    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"
Nov 01 03:42:35.045162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.045105    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" event={"ID":"015c1cd8-f0e0-4ccf-8719-51b08452764e","Type":"ContainerDied","Data":"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"}
Nov 01 03:42:35.045162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.045141    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h" event={"ID":"015c1cd8-f0e0-4ccf-8719-51b08452764e","Type":"ContainerDied","Data":"407034ff19173ebb00b849556a3456c2c9b7f0b948914e4cfb0b54b88758ff15"}
Nov 01 03:42:35.053166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.053143    4892 scope.go:122] "RemoveContainer" containerID="a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"
Nov 01 03:42:35.064843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.064821    4892 scope.go:122] "RemoveContainer" containerID="a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"
Nov 01 03:42:35.065270 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:35.065248    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc\": container with ID starting with a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc not found: ID does not exist" containerID="a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"
Nov 01 03:42:35.065325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.065281    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc"} err="failed to get container status \"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc\": rpc error: code = NotFound desc = could not find container \"a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc\": container with ID starting with a9315b9fbd111441652e48e616997c46a06c9a52718138eab90339dd607a2cbc not found: ID does not exist"
Nov 01 03:42:35.065325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.065300    4892 scope.go:122] "RemoveContainer" containerID="27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"
Nov 01 03:42:35.077235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.077216    4892 scope.go:122] "RemoveContainer" containerID="27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"
Nov 01 03:42:35.077544 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:35.077525    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca\": container with ID starting with 27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca not found: ID does not exist" containerID="27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"
Nov 01 03:42:35.077600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.077555    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca"} err="failed to get container status \"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca\": rpc error: code = NotFound desc = could not find container \"27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca\": container with ID starting with 27073e0e34160409ef5ad686c6ac87e38768298732b4b584d0555dce448befca not found: ID does not exist"
Nov 01 03:42:35.219119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.219037    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:42:35.268876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.268844    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-l5kq7"]
Nov 01 03:42:35.343696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.343661    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:42:35.351020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.350967    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-438641-openshift-default-974f6f5d5-wk58h"]
Nov 01 03:42:35.461914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.461880    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:42:35.475133 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.475054    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-44mwv"]
Nov 01 03:42:35.702724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.702691    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" path="/var/lib/kubelet/pods/015c1cd8-f0e0-4ccf-8719-51b08452764e/volumes"
Nov 01 03:42:35.703188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.703174    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" path="/var/lib/kubelet/pods/61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1/volumes"
Nov 01 03:42:35.703591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:35.703579    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" path="/var/lib/kubelet/pods/e08c0f6c-80eb-49ea-b362-07767a84306c/volumes"
Nov 01 03:42:37.212191 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212146    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7081-6c873a"]
Nov 01 03:42:37.212846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212823    4892 cpu_manager.go:414] "containerMap: removing container" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212849    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212882    4892 cpu_manager.go:414] "containerMap: removing container" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212890    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212906    4892 cpu_manager.go:414] "containerMap: removing container" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212914    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212938    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery"
Nov 01 03:42:37.212960 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212946    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212970    4892 cpu_manager.go:414] "containerMap: removing container" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.212994    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.213126    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.213140    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="61942bc5-8d1f-4fa9-afeb-6c4a051a4cc1" containerName="istio-proxy"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.213153    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e08c0f6c-80eb-49ea-b362-07767a84306c" containerName="discovery"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.213163    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="fec7bfac-b2a1-4c3d-8726-5f3f2e78ce97" containerName="istio-proxy"
Nov 01 03:42:37.213323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.213176    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="015c1cd8-f0e0-4ccf-8719-51b08452764e" containerName="istio-proxy"
Nov 01 03:42:37.217873 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.217850    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:37.269727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.269692    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7081-6c873a"]
Nov 01 03:42:37.315732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.315702    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vpgz\" (UniqueName: \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\") pod \"test-curl-7081-6c873a\" (UID: \"ff1672c1-076c-45f2-8362-662525f7265f\") " pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:37.316335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.316312    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-h2kzj\""
Nov 01 03:42:37.417212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.417171    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vpgz\" (UniqueName: \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\") pod \"test-curl-7081-6c873a\" (UID: \"ff1672c1-076c-45f2-8362-662525f7265f\") " pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:37.481417 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.481337    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vpgz\" (UniqueName: \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\") pod \"test-curl-7081-6c873a\" (UID: \"ff1672c1-076c-45f2-8362-662525f7265f\") " pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:37.529684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.529646    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:37.907033 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.906998    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7081-6c873a"]
Nov 01 03:42:37.909078 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:37.909040    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff1672c1_076c_45f2_8362_662525f7265f.slice/crio-3f138d85c5aeb9b785af3e036321a943cc82986caa59f2a3940af341df3a801d WatchSource:0}: Error finding container 3f138d85c5aeb9b785af3e036321a943cc82986caa59f2a3940af341df3a801d: Status 404 returned error can't find the container with id 3f138d85c5aeb9b785af3e036321a943cc82986caa59f2a3940af341df3a801d
Nov 01 03:42:37.911076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:37.911057    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 03:42:38.066389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:38.066354    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7081-6c873a" event={"ID":"ff1672c1-076c-45f2-8362-662525f7265f","Type":"ContainerStarted","Data":"3f138d85c5aeb9b785af3e036321a943cc82986caa59f2a3940af341df3a801d"}
Nov 01 03:42:40.357144 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.357098    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:42:40.370120 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.369289    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458132 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458099    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458138    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458171    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458271    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458297    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458421    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458529    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp84z\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.458708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458613    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.459008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.458842    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.465540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.465275    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:42:40.476297 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.476263    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-d84de6-openshift-default-dockercfg-w42tv\""
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561319    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561385    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561421    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561460    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561488    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561511    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561566    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561638    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp84z\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.561689    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.562627    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.562914    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.563228    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.565348    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.568001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.566417    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.640096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.635550    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.640096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.636142    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.640096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.636933    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.640096 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.637535    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp84z\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\") pod \"e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") " pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:40.690914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:40.690885    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:41.018469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.018422    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:42:41.018766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.018685    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" podUID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" containerName="maas-api" containerID="cri-o://bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e" gracePeriod=30
Nov 01 03:42:41.092695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.092646    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7081-6c873a" event={"ID":"ff1672c1-076c-45f2-8362-662525f7265f","Type":"ContainerStarted","Data":"80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16"}
Nov 01 03:42:41.194185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.194146    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:42:41.245008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.244938    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:41.245291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.245264    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:41.245426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.245409    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:41.259260 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.257328    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7081-6c873a" podStartSLOduration=1.489187422 podStartE2EDuration="4.257308356s" totalImagesPullingTime="2.768120934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:41.25623392 +0000 UTC m=+2127.672133450" watchObservedRunningTime="2026-11-01 03:42:41.257308356 +0000 UTC m=+2127.673207888"
Nov 01 03:42:41.335408 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.335363    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:42:41.484581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.484550    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\") pod \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") "
Nov 01 03:42:41.485040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.484716    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\") pod \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") "
Nov 01 03:42:41.485040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.484734    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") pod \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") "
Nov 01 03:42:41.485040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.484767    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\" (UniqueName: \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\") pod \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\" (UID: \"97b2f1c3-b3a0-48cd-99d7-2fb944317425\") "
Nov 01 03:42:41.485481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.485454    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca" pod "97b2f1c3-b3a0-48cd-99d7-2fb944317425" (UID: "97b2f1c3-b3a0-48cd-99d7-2fb944317425"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:42:41.487320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.487288    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls" pod "97b2f1c3-b3a0-48cd-99d7-2fb944317425" (UID: "97b2f1c3-b3a0-48cd-99d7-2fb944317425"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:42:41.487405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.487297    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs" pod "97b2f1c3-b3a0-48cd-99d7-2fb944317425" (UID: "97b2f1c3-b3a0-48cd-99d7-2fb944317425"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:42:41.488573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.488553    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs" pod "97b2f1c3-b3a0-48cd-99d7-2fb944317425" (UID: "97b2f1c3-b3a0-48cd-99d7-2fb944317425"). InnerVolumeSpecName "kube-api-access-w68zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:41.589200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.588954    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/97b2f1c3-b3a0-48cd-99d7-2fb944317425-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:41.589200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.589034    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:41.589200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.589053    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w68zs\" (UniqueName: \"kubernetes.io/projected/97b2f1c3-b3a0-48cd-99d7-2fb944317425-kube-api-access-w68zs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:41.589200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:41.589070    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97b2f1c3-b3a0-48cd-99d7-2fb944317425-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:42.115004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.107283    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerStarted","Data":"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"}
Nov 01 03:42:42.115004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.107355    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerStarted","Data":"a1b8fd0da77dfab2f104e42eb65534bfbf371bae1881fdfbcb77125248053aae"}
Nov 01 03:42:42.131002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.130552    4892 generic.go:356] "Generic (PLEG): container finished" podID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" containerID="bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e" exitCode=0
Nov 01 03:42:42.150003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.131380    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"
Nov 01 03:42:42.150003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.132186    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" event={"ID":"97b2f1c3-b3a0-48cd-99d7-2fb944317425","Type":"ContainerDied","Data":"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"}
Nov 01 03:42:42.150003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.132243    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx" event={"ID":"97b2f1c3-b3a0-48cd-99d7-2fb944317425","Type":"ContainerDied","Data":"5540ac6c48d0759686cef01a8bdf050c314b80c82a7f6ac9df2819cebe870d90"}
Nov 01 03:42:42.150003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.132273    4892 scope.go:122] "RemoveContainer" containerID="bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"
Nov 01 03:42:42.173320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.169376    4892 scope.go:122] "RemoveContainer" containerID="bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"
Nov 01 03:42:42.173320 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:42.169884    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e\": container with ID starting with bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e not found: ID does not exist" containerID="bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"
Nov 01 03:42:42.173320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.169926    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e"} err="failed to get container status \"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e\": rpc error: code = NotFound desc = could not find container \"bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e\": container with ID starting with bd57ec8b0c4473ee22f29371bad55055243b7fb76fba3d6600e68ba832c03f8e not found: ID does not exist"
Nov 01 03:42:42.238662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.238605    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podStartSLOduration=2.238590721 podStartE2EDuration="2.238590721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:42.234657592 +0000 UTC m=+2128.650557121" watchObservedRunningTime="2026-11-01 03:42:42.238590721 +0000 UTC m=+2128.654490249"
Nov 01 03:42:42.348728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.348695    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:42:42.352364 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.352328    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-686153-695dcd7cd6-lwhvx"]
Nov 01 03:42:42.691998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.691949    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:42.694484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.694411    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:42:42.695762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.695740    4892 cpu_manager.go:414] "containerMap: removing container" podUID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" containerName="maas-api"
Nov 01 03:42:42.695762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.695761    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" containerName="maas-api"
Nov 01 03:42:42.695911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.695874    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" containerName="maas-api"
Nov 01 03:42:42.700334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.700316    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:42.700420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.700396    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.737458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.737431    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7081-6c873a"]
Nov 01 03:42:42.742233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.742201    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7081-6c873a"]
Nov 01 03:42:42.765363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.765334    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-2c8373-serving-cert\""
Nov 01 03:42:42.768875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.768837    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:42:42.808085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.808050    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.808291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.808162    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.808291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.808207    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.808291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.808237    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdglw\" (UniqueName: \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.884155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.884117    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7081-670575"]
Nov 01 03:42:42.884641 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.884628    4892 cpu_manager.go:414] "containerMap: removing container" podUID="ff1672c1-076c-45f2-8362-662525f7265f" containerName="test-curl-7081-6c873a"
Nov 01 03:42:42.884692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.884643    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff1672c1-076c-45f2-8362-662525f7265f" containerName="test-curl-7081-6c873a"
Nov 01 03:42:42.884748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.884738    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff1672c1-076c-45f2-8362-662525f7265f" containerName="test-curl-7081-6c873a"
Nov 01 03:42:42.888202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.888182    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:42.909133 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.909100    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.909335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.909163    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdglw\" (UniqueName: \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.909398 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.909330    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.909453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.909417    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.909573 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:42.909552    4892 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-serving-cert: secret "maas-api-e2e-api-a-2c8373-serving-cert" not found
Nov 01 03:42:42.909644 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:42.909616    4892 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls podName:77324486-1a26-460b-8282-3cbfcfefa919 nodeName:}" failed. No retries permitted until 2026-11-01 03:42:43.40959799 +0000 UTC m=+2129.825497515 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls") pod "maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" (UID: "77324486-1a26-460b-8282-3cbfcfefa919") : secret "maas-api-e2e-api-a-2c8373-serving-cert" not found
Nov 01 03:42:42.910155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.910132    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.911783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.911753    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:42.946816 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.943007    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7081-670575"]
Nov 01 03:42:42.985108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:42.977365    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdglw\" (UniqueName: \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:43.011025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.010970    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtlrd\" (UniqueName: \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\") pod \"test-curl-7081-670575\" (UID: \"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a\") " pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:43.112052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.112018    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtlrd\" (UniqueName: \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\") pod \"test-curl-7081-670575\" (UID: \"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a\") " pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:43.137893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.137856    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7081-6c873a" podUID="ff1672c1-076c-45f2-8362-662525f7265f" containerName="test-curl-7081-6c873a" containerID="cri-o://80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16" gracePeriod=2
Nov 01 03:42:43.138161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.138141    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:43.139035 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.139011    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:42:43.173307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.173278    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtlrd\" (UniqueName: \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\") pod \"test-curl-7081-670575\" (UID: \"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a\") " pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:43.189567 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:43.189532    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-6c873a\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="ff1672c1-076c-45f2-8362-662525f7265f" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:43.201032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.200941    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:43.241157 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:43.241119    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-6c873a\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="ff1672c1-076c-45f2-8362-662525f7265f" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:43.417228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.417188    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:43.419719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.419688    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") pod \"maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:43.579214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.579187    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7081-670575"]
Nov 01 03:42:43.581683 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:43.581658    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f0d6a99_689e_4a2d_a6c7_e4ce586c749a.slice/crio-c648bbbb430f5277932dd2b54289d9de70168b36f35d677d6e43e1f7a255940c WatchSource:0}: Error finding container c648bbbb430f5277932dd2b54289d9de70168b36f35d677d6e43e1f7a255940c: Status 404 returned error can't find the container with id c648bbbb430f5277932dd2b54289d9de70168b36f35d677d6e43e1f7a255940c
Nov 01 03:42:43.613919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.613891    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:43.702633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:43.702602    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97b2f1c3-b3a0-48cd-99d7-2fb944317425" path="/var/lib/kubelet/pods/97b2f1c3-b3a0-48cd-99d7-2fb944317425/volumes"
Nov 01 03:42:43.749086 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:43.749042    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-6c873a\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="ff1672c1-076c-45f2-8362-662525f7265f" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:44.047451 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:44.047421    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:42:44.049809 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:44.049785    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77324486_1a26_460b_8282_3cbfcfefa919.slice/crio-a7c5e854c2af0ca1b2ee72692d11ba5493e1af183627a5ed503e0967758a99f9 WatchSource:0}: Error finding container a7c5e854c2af0ca1b2ee72692d11ba5493e1af183627a5ed503e0967758a99f9: Status 404 returned error can't find the container with id a7c5e854c2af0ca1b2ee72692d11ba5493e1af183627a5ed503e0967758a99f9
Nov 01 03:42:44.145995 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:44.145951    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7081-670575" event={"ID":"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a","Type":"ContainerStarted","Data":"a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5"}
Nov 01 03:42:44.146185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:44.146022    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7081-670575" event={"ID":"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a","Type":"ContainerStarted","Data":"c648bbbb430f5277932dd2b54289d9de70168b36f35d677d6e43e1f7a255940c"}
Nov 01 03:42:44.147275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:44.147247    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" event={"ID":"77324486-1a26-460b-8282-3cbfcfefa919","Type":"ContainerStarted","Data":"a7c5e854c2af0ca1b2ee72692d11ba5493e1af183627a5ed503e0967758a99f9"}
Nov 01 03:42:44.260968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:44.260912    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7081-670575" podStartSLOduration=2.260898115 podStartE2EDuration="2.260898115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:44.259575602 +0000 UTC m=+2130.675475133" watchObservedRunningTime="2026-11-01 03:42:44.260898115 +0000 UTC m=+2130.676797643"
Nov 01 03:42:45.156370 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.156333    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" event={"ID":"77324486-1a26-460b-8282-3cbfcfefa919","Type":"ContainerStarted","Data":"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"}
Nov 01 03:42:45.156955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.156935    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:45.278636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.278575    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" podStartSLOduration=2.366293135 podStartE2EDuration="3.278556487s" totalImagesPullingTime="912.263352ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:45.277393521 +0000 UTC m=+2131.693293051" watchObservedRunningTime="2026-11-01 03:42:45.278556487 +0000 UTC m=+2131.694456018"
Nov 01 03:42:45.405624 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.405594    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:45.543584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.543504    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\" (UniqueName: \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\") pod \"ff1672c1-076c-45f2-8362-662525f7265f\" (UID: \"ff1672c1-076c-45f2-8362-662525f7265f\") "
Nov 01 03:42:45.545656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.545629    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz" pod "ff1672c1-076c-45f2-8362-662525f7265f" (UID: "ff1672c1-076c-45f2-8362-662525f7265f"). InnerVolumeSpecName "kube-api-access-7vpgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:45.644768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.644735    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vpgz\" (UniqueName: \"kubernetes.io/projected/ff1672c1-076c-45f2-8362-662525f7265f-kube-api-access-7vpgz\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:45.702203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.702171    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ff1672c1-076c-45f2-8362-662525f7265f" path="/var/lib/kubelet/pods/ff1672c1-076c-45f2-8362-662525f7265f/volumes"
Nov 01 03:42:45.774082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.774047    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7081-670575"]
Nov 01 03:42:45.780567 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:45.780530    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7081-670575"]
Nov 01 03:42:46.165440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.165405    4892 generic.go:356] "Generic (PLEG): container finished" podID="ff1672c1-076c-45f2-8362-662525f7265f" containerID="80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16" exitCode=137
Nov 01 03:42:46.165887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.165486    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-6c873a"
Nov 01 03:42:46.165887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.165515    4892 scope.go:122] "RemoveContainer" containerID="80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16"
Nov 01 03:42:46.165887 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.165660    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7081-670575" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" containerName="test-curl-7081-670575" containerID="cri-o://a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5" gracePeriod=2
Nov 01 03:42:46.175393 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.175374    4892 scope.go:122] "RemoveContainer" containerID="80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16"
Nov 01 03:42:46.175668 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:46.175646    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16\": container with ID starting with 80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16 not found: ID does not exist" containerID="80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16"
Nov 01 03:42:46.175729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.175677    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16"} err="failed to get container status \"80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16\": rpc error: code = NotFound desc = could not find container \"80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16\": container with ID starting with 80223d8f7c844809b7673ab4f3204a20799edcbebd413e6217277e0330717e16 not found: ID does not exist"
Nov 01 03:42:46.217222 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:46.217191    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:46.268866 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:46.268827    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:46.859001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.850636    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:42:46.859001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.851690    4892 cpu_manager.go:414] "containerMap: removing container" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" containerName="test-curl-7081-670575"
Nov 01 03:42:46.859001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.851714    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" containerName="test-curl-7081-670575"
Nov 01 03:42:46.859001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.851884    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" containerName="test-curl-7081-670575"
Nov 01 03:42:46.870999 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.866080    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.927540 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.927289    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-2906cb-openshift-default-dockercfg-2hgsv\""
Nov 01 03:42:46.927540 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:46.927390    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:46.962656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.958621    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:42:46.994472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.985429    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn58h\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.995141 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.995106    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.995330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.995312    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.995519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.995503    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.995693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.995677    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.995921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.995906    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.996091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.996073    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.996201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.996189    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:46.996299 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:46.996288    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.100243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.100202    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.100591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.100561    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.100921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.100898    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101085    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101293    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101486    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101675    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn58h\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101837 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101820    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.101961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101944    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.102212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.102005    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.102336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.101426    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.102785 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.102765    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.103188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.103155    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.103312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.103184    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.117122 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.116914    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.117969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.117949    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.164196 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.164163    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.165088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.165063    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn58h\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\") pod \"e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") " pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.182478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.182449    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:42:47.187722 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.187697    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.239435 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:47.239396    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:47.242172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.242105    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-5fd6cf\""
Nov 01 03:42:47.242312 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.242210    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-5fd6cf-dockercfg-hjjsx\""
Nov 01 03:42:47.245229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.245205    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:42:47.249555 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.249524    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:47.330003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.322700    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gshj\" (UniqueName: \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.330003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.322794    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.388726 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.388636    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:42:47.395406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.395376    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.407119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.406239    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:42:47.426265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.423940    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gshj\" (UniqueName: \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.426265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.424042    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.426265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.425078    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.488219 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.486802    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gshj\" (UniqueName: \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\") pod \"payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.504709 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.504242    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:47.527558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.527507    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcbjn\" (UniqueName: \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.527693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.527572    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.567197 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:47.567151    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:47.629097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.629066    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcbjn\" (UniqueName: \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.629270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.629118    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.629884 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.629857    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.690480 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.690402    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcbjn\" (UniqueName: \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\") pod \"payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") " pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.730156 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.730114    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:47.834809 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.834772    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:42:47.836957 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:47.836922    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4fc747_c727_4424_9ed9_7ab87724471d.slice/crio-e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6 WatchSource:0}: Error finding container e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6: Status 404 returned error can't find the container with id e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6
Nov 01 03:42:47.841148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.840305    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:47.841148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.840366    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:47.841148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.840396    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:42:47.930711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:47.930668    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:42:47.934849 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:47.934820    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cf599c0_4906_496b_9b34_ad6c0175e3c0.slice/crio-da28506ef567729db0f5575cbc69db127f0cefacd86221bddb2d9b5ed25e43fd WatchSource:0}: Error finding container da28506ef567729db0f5575cbc69db127f0cefacd86221bddb2d9b5ed25e43fd: Status 404 returned error can't find the container with id da28506ef567729db0f5575cbc69db127f0cefacd86221bddb2d9b5ed25e43fd
Nov 01 03:42:48.126626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.125919    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:42:48.130010 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:48.129938    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4efd5f84_128f_4995_896a_ba381ada2b58.slice/crio-a0a4403389ed75acf4b5977ac888a19b6b1b81ee1b5e7d3b227105c9c88833cf WatchSource:0}: Error finding container a0a4403389ed75acf4b5977ac888a19b6b1b81ee1b5e7d3b227105c9c88833cf: Status 404 returned error can't find the container with id a0a4403389ed75acf4b5977ac888a19b6b1b81ee1b5e7d3b227105c9c88833cf
Nov 01 03:42:48.201509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.201411    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerStarted","Data":"7812abc0e90c4a291674d18e199706075e808f22cf180f0629480f538d516b27"}
Nov 01 03:42:48.201509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.201480    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerStarted","Data":"e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6"}
Nov 01 03:42:48.205131 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.204770    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" event={"ID":"4efd5f84-128f-4995-896a-ba381ada2b58","Type":"ContainerStarted","Data":"a0a4403389ed75acf4b5977ac888a19b6b1b81ee1b5e7d3b227105c9c88833cf"}
Nov 01 03:42:48.207024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.206995    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" event={"ID":"6cf599c0-4906-496b-9b34-ad6c0175e3c0","Type":"ContainerStarted","Data":"26ddf1cd5d2cd74972eb2ef9c2e77735c09f7f56514c4af0db5c0ec7f36342c2"}
Nov 01 03:42:48.207151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.207051    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" event={"ID":"6cf599c0-4906-496b-9b34-ad6c0175e3c0","Type":"ContainerStarted","Data":"da28506ef567729db0f5575cbc69db127f0cefacd86221bddb2d9b5ed25e43fd"}
Nov 01 03:42:48.207832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.207804    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:42:48.252002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.250287    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:48.252002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.251819    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:42:48.252002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.251865    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:42:48.325151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.325097    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podStartSLOduration=2.325078911 podStartE2EDuration="2.325078911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:48.32313561 +0000 UTC m=+2134.739035140" watchObservedRunningTime="2026-11-01 03:42:48.325078911 +0000 UTC m=+2134.740978439"
Nov 01 03:42:48.431664 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.431635    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:48.441938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.441887    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" podStartSLOduration=1.441869629 podStartE2EDuration="1.441869629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:48.439316239 +0000 UTC m=+2134.855215769" watchObservedRunningTime="2026-11-01 03:42:48.441869629 +0000 UTC m=+2134.857769158"
Nov 01 03:42:48.548466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.548430    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\" (UniqueName: \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\") pod \"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a\" (UID: \"1f0d6a99-689e-4a2d-a6c7-e4ce586c749a\") "
Nov 01 03:42:48.550690 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.550606    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd" pod "1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" (UID: "1f0d6a99-689e-4a2d-a6c7-e4ce586c749a"). InnerVolumeSpecName "kube-api-access-vtlrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:42:48.651136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:48.651098    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtlrd\" (UniqueName: \"kubernetes.io/projected/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a-kube-api-access-vtlrd\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:42:49.214006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.213946    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" event={"ID":"4efd5f84-128f-4995-896a-ba381ada2b58","Type":"ContainerStarted","Data":"c56a4ee6dd66af157e6d4ec1fbf6984c9855511e1601f63ce131103ba7f5df9b"}
Nov 01 03:42:49.214419 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.214042    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:42:49.215532 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.215507    4892 generic.go:356] "Generic (PLEG): container finished" podID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" containerID="a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5" exitCode=137
Nov 01 03:42:49.215660 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.215540    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:49.215660 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.215605    4892 scope.go:122] "RemoveContainer" containerID="a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5"
Nov 01 03:42:49.225042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.225023    4892 scope.go:122] "RemoveContainer" containerID="a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5"
Nov 01 03:42:49.225298 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:49.225279    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5\": container with ID starting with a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5 not found: ID does not exist" containerID="a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5"
Nov 01 03:42:49.225361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.225306    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5"} err="failed to get container status \"a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5\": rpc error: code = NotFound desc = could not find container \"a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5\": container with ID starting with a3d44957dbaf63a76c6ddcaaab1cf48faf4a109db5cfaae9796bb7c4664820c5 not found: ID does not exist"
Nov 01 03:42:49.255300 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.255279    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:49.334489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.334439    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" podStartSLOduration=2.3344239399999998 podStartE2EDuration="2.33442394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:49.332406446 +0000 UTC m=+2135.748305987" watchObservedRunningTime="2026-11-01 03:42:49.33442394 +0000 UTC m=+2135.750323472"
Nov 01 03:42:49.383810 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:49.383715    4892 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7081-670575\" is forbidden: User \"system:node:ip-10-0-134-33.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-134-33.us-west-1.compute.internal' and this object" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" pod="opendatahub/test-curl-7081-670575"
Nov 01 03:42:49.701940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:49.701859    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f0d6a99-689e-4a2d-a6c7-e4ce586c749a" path="/var/lib/kubelet/pods/1f0d6a99-689e-4a2d-a6c7-e4ce586c749a/volumes"
Nov 01 03:42:50.223237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:50.223207    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:50.224081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:50.224059    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:42:52.179798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:52.179771    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:42:53.943124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:53.943087    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:42:53.948366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:53.948344    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.006155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.006122    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:42:54.065013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.053280    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d84de6\""
Nov 01 03:42:54.080200 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.069217    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d84de6-dockercfg-mhtrm\""
Nov 01 03:42:54.111294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.093265    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:42:54.150385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.150352    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.159535 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.159497    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.164641 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.159562    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6lcl\" (UniqueName: \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.164641 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.162432    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf6rq\" (UniqueName: \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.164641 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.162576    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.169581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.169555    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:42:54.264135 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264102    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.264324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264218    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.264324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264253    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6lcl\" (UniqueName: \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.264446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264363    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf6rq\" (UniqueName: \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.264947 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264924    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.265053 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.264934    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.372002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.354758    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6lcl\" (UniqueName: \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\") pod \"payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") " pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.372002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.366429    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf6rq\" (UniqueName: \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\") pod \"payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.480603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.480567    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:54.561767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.561739    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:54.860111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.860083    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:42:54.861634 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:54.861605    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba86e65_6371_4c76_9e31_90badb06dd05.slice/crio-ca314268e5cc5636313fcae89b45473c138fdd3f113d0dc776d7a47945b1a1be WatchSource:0}: Error finding container ca314268e5cc5636313fcae89b45473c138fdd3f113d0dc776d7a47945b1a1be: Status 404 returned error can't find the container with id ca314268e5cc5636313fcae89b45473c138fdd3f113d0dc776d7a47945b1a1be
Nov 01 03:42:54.973150 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:54.973127    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:42:54.974648 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:42:54.974623    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd6f6bba_82b3_44b5_a686_7a6eb4341633.slice/crio-4d413363fa9507de60eedb846bf1210a251e19eb0ded61f42d933b04bfd10a6d WatchSource:0}: Error finding container 4d413363fa9507de60eedb846bf1210a251e19eb0ded61f42d933b04bfd10a6d: Status 404 returned error can't find the container with id 4d413363fa9507de60eedb846bf1210a251e19eb0ded61f42d933b04bfd10a6d
Nov 01 03:42:55.256026 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.255973    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" event={"ID":"cd6f6bba-82b3-44b5-a686-7a6eb4341633","Type":"ContainerStarted","Data":"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"}
Nov 01 03:42:55.256186 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.256053    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" event={"ID":"cd6f6bba-82b3-44b5-a686-7a6eb4341633","Type":"ContainerStarted","Data":"4d413363fa9507de60eedb846bf1210a251e19eb0ded61f42d933b04bfd10a6d"}
Nov 01 03:42:55.256259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.256217    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:42:55.257776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.257750    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" event={"ID":"dba86e65-6371-4c76-9e31-90badb06dd05","Type":"ContainerStarted","Data":"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"}
Nov 01 03:42:55.257911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.257790    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" event={"ID":"dba86e65-6371-4c76-9e31-90badb06dd05","Type":"ContainerStarted","Data":"ca314268e5cc5636313fcae89b45473c138fdd3f113d0dc776d7a47945b1a1be"}
Nov 01 03:42:55.257911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.257814    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:42:55.373756 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.373705    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" podStartSLOduration=2.373689916 podStartE2EDuration="2.373689916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:55.372327535 +0000 UTC m=+2141.788227061" watchObservedRunningTime="2026-11-01 03:42:55.373689916 +0000 UTC m=+2141.789589444"
Nov 01 03:42:55.493864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:42:55.493814    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" podStartSLOduration=1.493799642 podStartE2EDuration="1.493799642s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:42:55.4910206 +0000 UTC m=+2141.906920133" watchObservedRunningTime="2026-11-01 03:42:55.493799642 +0000 UTC m=+2141.909699211"
Nov 01 03:42:57.612303 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:42:57.612263    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:43:00.226098 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:00.226070    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:43:00.226577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:00.226143    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:43:03.661924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.661894    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:43:03.666082 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.666063    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.733166 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.733133    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:43:03.792436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.792394    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-2906cb\""
Nov 01 03:43:03.835587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.828607    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-2906cb-dockercfg-d5794\""
Nov 01 03:43:03.835587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.831997    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22rg7\" (UniqueName: \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.835587 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.835325    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.847629 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.847597    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:43:03.852529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.852502    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:03.877237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.877208    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:43:03.936571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.936490    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.936737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.936573    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:03.936737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.936612    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq5rb\" (UniqueName: \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:03.936826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.936767    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22rg7\" (UniqueName: \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.937296 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.937279    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:03.998039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:03.997999    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22rg7\" (UniqueName: \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\") pod \"payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:04.038266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.038229    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:04.038440 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.038294    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq5rb\" (UniqueName: \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:04.039036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.039014    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:04.099700 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.099668    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq5rb\" (UniqueName: \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\") pod \"payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") " pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:04.168884 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.168847    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:04.280637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.280603    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:04.552091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.552061    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:43:04.555084 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:43:04.555061    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88f7fedd_f6eb_4cca_9f39_f30233f6fb90.slice/crio-c5c182f3394297d134f02d79853b81f4c193b22de8ba06c18f6b436af0607235 WatchSource:0}: Error finding container c5c182f3394297d134f02d79853b81f4c193b22de8ba06c18f6b436af0607235: Status 404 returned error can't find the container with id c5c182f3394297d134f02d79853b81f4c193b22de8ba06c18f6b436af0607235
Nov 01 03:43:04.663541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:04.663516    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:43:04.665183 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:43:04.665159    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4a54616_901a_4755_acf9_dbae83adc1ab.slice/crio-1f17a1c303511aea9751ebdde197554c38ab021b6e0ab217c30f2a6aed03cc29 WatchSource:0}: Error finding container 1f17a1c303511aea9751ebdde197554c38ab021b6e0ab217c30f2a6aed03cc29: Status 404 returned error can't find the container with id 1f17a1c303511aea9751ebdde197554c38ab021b6e0ab217c30f2a6aed03cc29
Nov 01 03:43:05.338545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.338506    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" event={"ID":"88f7fedd-f6eb-4cca-9f39-f30233f6fb90","Type":"ContainerStarted","Data":"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"}
Nov 01 03:43:05.338758 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.338575    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:05.338758 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.338601    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" event={"ID":"88f7fedd-f6eb-4cca-9f39-f30233f6fb90","Type":"ContainerStarted","Data":"c5c182f3394297d134f02d79853b81f4c193b22de8ba06c18f6b436af0607235"}
Nov 01 03:43:05.340175 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.340147    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" event={"ID":"a4a54616-901a-4755-acf9-dbae83adc1ab","Type":"ContainerStarted","Data":"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"}
Nov 01 03:43:05.340302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.340198    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" event={"ID":"a4a54616-901a-4755-acf9-dbae83adc1ab","Type":"ContainerStarted","Data":"1f17a1c303511aea9751ebdde197554c38ab021b6e0ab217c30f2a6aed03cc29"}
Nov 01 03:43:05.340302 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.340226    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:05.455536 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.455479    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" podStartSLOduration=2.455460751 podStartE2EDuration="2.455460751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:43:05.454740671 +0000 UTC m=+2151.870640213" watchObservedRunningTime="2026-11-01 03:43:05.455460751 +0000 UTC m=+2151.871360281"
Nov 01 03:43:05.591538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:05.591390    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" podStartSLOduration=2.591370888 podStartE2EDuration="2.591370888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:43:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:43:05.573319136 +0000 UTC m=+2151.989218667" watchObservedRunningTime="2026-11-01 03:43:05.591370888 +0000 UTC m=+2152.007270418"
Nov 01 03:43:06.266716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:06.266688    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:43:06.267248 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:06.266767    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:43:07.682436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:07.682397    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:43:07.732436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:07.732403    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:43:15.059557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:15.059519    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:43:16.349909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:16.349882    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:43:16.350328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:16.349954    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:43:20.059870 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:20.059811    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing" containerID="cri-o://c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303" gracePeriod=25
Nov 01 03:43:22.704908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:22.704826    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main" containerID="cri-o://efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1" gracePeriod=45
Nov 01 03:43:22.752862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:22.752825    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main" containerID="cri-o://c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa" gracePeriod=45
Nov 01 03:43:23.027184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.027154    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:43:23.050055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.050033    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:43:23.090578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090548    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\" (UniqueName: \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090586    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090681    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090760    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090818    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090878    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\") pod \"1e2b14e7-6838-4c36-95c6-38526a72758d\" (UID: \"1e2b14e7-6838-4c36-95c6-38526a72758d\") "
Nov 01 03:43:23.090915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.090883    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.091105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.091084    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.091208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.091184    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.091379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.091363    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.091379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.091380    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.091514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.091391    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.093003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.092971    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:43:23.093213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.093192    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.093213 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.093206    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9" pod "1e2b14e7-6838-4c36-95c6-38526a72758d" (UID: "1e2b14e7-6838-4c36-95c6-38526a72758d"). InnerVolumeSpecName "kube-api-access-9zdf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:43:23.191959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.191922    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192078    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\" (UniqueName: \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192169    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192201    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192224    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.192329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192315    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192410    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\" (UniqueName: \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\") pod \"336471d8-7aed-45c6-9695-1231ecdf5eb6\" (UID: \"336471d8-7aed-45c6-9695-1231ecdf5eb6\") "
Nov 01 03:43:23.192630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192608    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.192991 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.192959    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.193085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.193001    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zdf9\" (UniqueName: \"kubernetes.io/projected/1e2b14e7-6838-4c36-95c6-38526a72758d-kube-api-access-9zdf9\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.193085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.193016    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e2b14e7-6838-4c36-95c6-38526a72758d-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.193085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.193029    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.193085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.193041    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e2b14e7-6838-4c36-95c6-38526a72758d-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.193270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.193141    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.197227 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.194824    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:43:23.197864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.197842    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:43:23.198151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.198127    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr" pod "336471d8-7aed-45c6-9695-1231ecdf5eb6" (UID: "336471d8-7aed-45c6-9695-1231ecdf5eb6"). InnerVolumeSpecName "kube-api-access-6nkcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:43:23.294257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.294213    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nkcr\" (UniqueName: \"kubernetes.io/projected/336471d8-7aed-45c6-9695-1231ecdf5eb6-kube-api-access-6nkcr\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.294257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.294253    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/336471d8-7aed-45c6-9695-1231ecdf5eb6-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.294257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.294263    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.294639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.294271    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/336471d8-7aed-45c6-9695-1231ecdf5eb6-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:23.470902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.470870    4892 generic.go:356] "Generic (PLEG): container finished" podID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerID="c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa" exitCode=0
Nov 01 03:43:23.471108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.470915    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"
Nov 01 03:43:23.471108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.470946    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" event={"ID":"336471d8-7aed-45c6-9695-1231ecdf5eb6","Type":"ContainerDied","Data":"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"}
Nov 01 03:43:23.471108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.471014    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k" event={"ID":"336471d8-7aed-45c6-9695-1231ecdf5eb6","Type":"ContainerDied","Data":"96eca9621c845eaabac0792a8c67fe68bc37145f3eddf75c30f4864ff18334b2"}
Nov 01 03:43:23.471108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.471037    4892 scope.go:122] "RemoveContainer" containerID="c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"
Nov 01 03:43:23.472744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.472714    4892 generic.go:356] "Generic (PLEG): container finished" podID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerID="efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1" exitCode=0
Nov 01 03:43:23.472859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.472805    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" event={"ID":"1e2b14e7-6838-4c36-95c6-38526a72758d","Type":"ContainerDied","Data":"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"}
Nov 01 03:43:23.472859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.472831    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp" event={"ID":"1e2b14e7-6838-4c36-95c6-38526a72758d","Type":"ContainerDied","Data":"33fdfc2ad0697150a7d5a2e5f596c85dc4c3b566ba272e7cfb23650d5f1b2863"}
Nov 01 03:43:23.473015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.472877    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"
Nov 01 03:43:23.480961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.480942    4892 scope.go:122] "RemoveContainer" containerID="c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"
Nov 01 03:43:23.481259 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:43:23.481238    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa\": container with ID starting with c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa not found: ID does not exist" containerID="c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"
Nov 01 03:43:23.481347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.481271    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa"} err="failed to get container status \"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa\": rpc error: code = NotFound desc = could not find container \"c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa\": container with ID starting with c00e640855c07b61c2032f1f41763164f21bef5031cdbc0b3801ffc7ce9408fa not found: ID does not exist"
Nov 01 03:43:23.481347 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.481295    4892 scope.go:122] "RemoveContainer" containerID="efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"
Nov 01 03:43:23.489914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.489897    4892 scope.go:122] "RemoveContainer" containerID="efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"
Nov 01 03:43:23.490186 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:43:23.490171    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1\": container with ID starting with efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1 not found: ID does not exist" containerID="efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"
Nov 01 03:43:23.490248 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.490192    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1"} err="failed to get container status \"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1\": rpc error: code = NotFound desc = could not find container \"efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1\": container with ID starting with efec411cbbd2fa94bcb3b3f684824b35d8dbd9d20d5d636b1971130d0ec675f1 not found: ID does not exist"
Nov 01 03:43:23.646630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.646552    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:43:23.654021 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.653973    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/simulator-w4-e8c789-kserve-ccc898c4c-vm69k"]
Nov 01 03:43:23.702387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.702351    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" path="/var/lib/kubelet/pods/336471d8-7aed-45c6-9695-1231ecdf5eb6/volumes"
Nov 01 03:43:23.765818 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.765788    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:43:23.770174 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:23.770149    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-e8c789/distinct-w4-e8c789-kserve-8fcb884b5-8hfsp"]
Nov 01 03:43:25.702221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:25.702189    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" path="/var/lib/kubelet/pods/1e2b14e7-6838-4c36-95c6-38526a72758d/volumes"
Nov 01 03:43:27.612157 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:43:27.612119    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:43:28.494522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:28.494486    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:43:38.494760 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:38.494718    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:43:40.318161 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.318101    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:43:40.406283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.406256    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\") pod \"e81a157c-9621-452c-a566-465756cab80a\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") "
Nov 01 03:43:40.406386 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.406369    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\" (UniqueName: \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\") pod \"e81a157c-9621-452c-a566-465756cab80a\" (UID: \"e81a157c-9621-452c-a566-465756cab80a\") "
Nov 01 03:43:40.406701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.406680    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume" pod "e81a157c-9621-452c-a566-465756cab80a" (UID: "e81a157c-9621-452c-a566-465756cab80a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:43:40.406850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.406836    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e81a157c-9621-452c-a566-465756cab80a-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:40.408433 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.408416    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd" pod "e81a157c-9621-452c-a566-465756cab80a" (UID: "e81a157c-9621-452c-a566-465756cab80a"). InnerVolumeSpecName "kube-api-access-cpwwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:43:40.507794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.507762    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpwwd\" (UniqueName: \"kubernetes.io/projected/e81a157c-9621-452c-a566-465756cab80a-kube-api-access-cpwwd\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:43:40.579036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.578940    4892 generic.go:356] "Generic (PLEG): container finished" podID="e81a157c-9621-452c-a566-465756cab80a" containerID="c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303" exitCode=0
Nov 01 03:43:40.579036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.579005    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"
Nov 01 03:43:40.579036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.579006    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" event={"ID":"e81a157c-9621-452c-a566-465756cab80a","Type":"ContainerDied","Data":"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"}
Nov 01 03:43:40.579287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.579065    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn" event={"ID":"e81a157c-9621-452c-a566-465756cab80a","Type":"ContainerDied","Data":"06225f8293ed50300b35d2238c895215817ef4860b7118d1bb0c3f722f8e1e6a"}
Nov 01 03:43:40.579287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.579094    4892 scope.go:122] "RemoveContainer" containerID="c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"
Nov 01 03:43:40.589268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.589242    4892 scope.go:122] "RemoveContainer" containerID="c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"
Nov 01 03:43:40.589531 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:43:40.589513    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303\": container with ID starting with c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303 not found: ID does not exist" containerID="c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"
Nov 01 03:43:40.589598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.589538    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303"} err="failed to get container status \"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303\": rpc error: code = NotFound desc = could not find container \"c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303\": container with ID starting with c2e9fb5aa2365421d8597acf9414c17f18dad5463a217772e9a8642bf190e303 not found: ID does not exist"
Nov 01 03:43:40.746964 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.746927    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:43:40.753345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:40.753317    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-e8c789-7679bdf579-zthqn"]
Nov 01 03:43:41.700824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:43:41.700794    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e81a157c-9621-452c-a566-465756cab80a" path="/var/lib/kubelet/pods/e81a157c-9621-452c-a566-465756cab80a/volumes"
Nov 01 03:43:57.610451 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:43:57.610414    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:44:02.785862 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.785823    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:44:02.786399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786384    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing"
Nov 01 03:44:02.786456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786401    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing"
Nov 01 03:44:02.786456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786421    4892 cpu_manager.go:414] "containerMap: removing container" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main"
Nov 01 03:44:02.786456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786427    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main"
Nov 01 03:44:02.786570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786468    4892 cpu_manager.go:414] "containerMap: removing container" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main"
Nov 01 03:44:02.786570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786473    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main"
Nov 01 03:44:02.786570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786548    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="336471d8-7aed-45c6-9695-1231ecdf5eb6" containerName="main"
Nov 01 03:44:02.786570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786557    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e81a157c-9621-452c-a566-465756cab80a" containerName="payload-pre-processing"
Nov 01 03:44:02.786570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.786564    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="1e2b14e7-6838-4c36-95c6-38526a72758d" containerName="main"
Nov 01 03:44:02.789962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.789938    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.860040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.859327    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-417aa8-openshift-default-dockercfg-4ctmq\""
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.897875    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.897926    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.897968    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898037    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfhbg\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898071    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898119    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898253    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898325    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.899043 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.898371    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.938041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.935085    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:44:02.999523 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999486    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999547    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999575    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999622    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999646    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999680    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999725    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfhbg\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999910 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999861    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:02.999975 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:02.999932    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.000064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.000045    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.000401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.000375    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.000545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.000499    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.000602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.000550    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.000696 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.000670    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.002346 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.002320    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.002463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.002412    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.061925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.061849    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.061925 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.061908    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfhbg\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\") pod \"e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") " pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.107085 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.107049    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:03.554145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.554118    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:44:03.557412 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:44:03.557382    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac89573_9530_4151_b20a_6e9e5ab79ef7.slice/crio-2c339d1f568ac9634ff3f3f75d9e24c5689b4f43db8afae3b228ee3cb8e05df5 WatchSource:0}: Error finding container 2c339d1f568ac9634ff3f3f75d9e24c5689b4f43db8afae3b228ee3cb8e05df5: Status 404 returned error can't find the container with id 2c339d1f568ac9634ff3f3f75d9e24c5689b4f43db8afae3b228ee3cb8e05df5
Nov 01 03:44:03.560073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.560042    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:03.560148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.560111    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:03.560148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.560142    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:03.726103 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.726069    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerStarted","Data":"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"}
Nov 01 03:44:03.726287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.726128    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerStarted","Data":"2c339d1f568ac9634ff3f3f75d9e24c5689b4f43db8afae3b228ee3cb8e05df5"}
Nov 01 03:44:03.858886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:03.858770    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podStartSLOduration=1.858750863 podStartE2EDuration="1.858750863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:44:03.854867638 +0000 UTC m=+2210.270767167" watchObservedRunningTime="2026-11-01 03:44:03.858750863 +0000 UTC m=+2210.274650393"
Nov 01 03:44:04.107180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.107147    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:04.113574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.113510    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:04.554322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.554282    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:44:04.554584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.554537    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" podUID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" containerName="maas-api" containerID="cri-o://31ee7372441d373c6ccf151adced0f212640503414ebf264c9dfeb4470fd8bf5" gracePeriod=30
Nov 01 03:44:04.741003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.737807    4892 generic.go:356] "Generic (PLEG): container finished" podID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" containerID="31ee7372441d373c6ccf151adced0f212640503414ebf264c9dfeb4470fd8bf5" exitCode=0
Nov 01 03:44:04.741003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.739127    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" event={"ID":"243c6dcb-5b19-42c9-a517-be3e3cb100a3","Type":"ContainerDied","Data":"31ee7372441d373c6ccf151adced0f212640503414ebf264c9dfeb4470fd8bf5"}
Nov 01 03:44:04.741003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.739188    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:04.756001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:04.741843    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:44:05.011810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.011781    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:44:05.196130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.196053    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\") pod \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") "
Nov 01 03:44:05.196130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.196109    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\") pod \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") "
Nov 01 03:44:05.196360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.196158    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\") pod \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") "
Nov 01 03:44:05.196577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.196534    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca" pod "243c6dcb-5b19-42c9-a517-be3e3cb100a3" (UID: "243c6dcb-5b19-42c9-a517-be3e3cb100a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:44:05.196751 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.196727    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\" (UniqueName: \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\") pod \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\" (UID: \"243c6dcb-5b19-42c9-a517-be3e3cb100a3\") "
Nov 01 03:44:05.197450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.197420    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/243c6dcb-5b19-42c9-a517-be3e3cb100a3-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:05.198515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.198488    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls" pod "243c6dcb-5b19-42c9-a517-be3e3cb100a3" (UID: "243c6dcb-5b19-42c9-a517-be3e3cb100a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:44:05.198839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.198819    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns" pod "243c6dcb-5b19-42c9-a517-be3e3cb100a3" (UID: "243c6dcb-5b19-42c9-a517-be3e3cb100a3"). InnerVolumeSpecName "kube-api-access-jrvns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:44:05.200175 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.200150    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs" pod "243c6dcb-5b19-42c9-a517-be3e3cb100a3" (UID: "243c6dcb-5b19-42c9-a517-be3e3cb100a3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:44:05.298538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.298498    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrvns\" (UniqueName: \"kubernetes.io/projected/243c6dcb-5b19-42c9-a517-be3e3cb100a3-kube-api-access-jrvns\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:05.298538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.298535    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:05.298748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.298551    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/243c6dcb-5b19-42c9-a517-be3e3cb100a3-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:05.796852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.796820    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"
Nov 01 03:44:05.797194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.797157    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv" event={"ID":"243c6dcb-5b19-42c9-a517-be3e3cb100a3","Type":"ContainerDied","Data":"51e399986ed6c20b2648da217fb60dfce660ead589520e1c3014098be6600ddd"}
Nov 01 03:44:05.797325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.797236    4892 scope.go:122] "RemoveContainer" containerID="31ee7372441d373c6ccf151adced0f212640503414ebf264c9dfeb4470fd8bf5"
Nov 01 03:44:05.963834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.963794    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:44:05.968107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:05.968080    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-f2383c-75d96bf4fb-mrqbv"]
Nov 01 03:44:07.702724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:07.702693    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" path="/var/lib/kubelet/pods/243c6dcb-5b19-42c9-a517-be3e3cb100a3/volumes"
Nov 01 03:44:13.083633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.083597    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:44:13.084229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.084167    4892 cpu_manager.go:414] "containerMap: removing container" podUID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" containerName="maas-api"
Nov 01 03:44:13.084229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.084181    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" containerName="maas-api"
Nov 01 03:44:13.084371 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.084289    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="243c6dcb-5b19-42c9-a517-be3e3cb100a3" containerName="maas-api"
Nov 01 03:44:13.089042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.088952    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.142102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.142045    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-417aa8\""
Nov 01 03:44:13.163160 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.163119    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwj5m\" (UniqueName: \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.163474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.163446    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.284814 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.284786    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-417aa8-dockercfg-djmjg\""
Nov 01 03:44:13.287322 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.286031    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwj5m\" (UniqueName: \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.287755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.287735    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.291411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.291199    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.298195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.298043    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:44:13.298195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.298115    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:44:13.303340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.303192    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.304968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.304943    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:44:13.348471 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.348408    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwj5m\" (UniqueName: \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\") pod \"payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.392208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.392180    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.392330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.392314    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lszdq\" (UniqueName: \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.403839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.403807    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:13.495661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.494068    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.495661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.494165    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lszdq\" (UniqueName: \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.495661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.495015    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.578576 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.578548    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lszdq\" (UniqueName: \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\") pod \"payload-processing-e2e-api-b-417aa8-84798965-r8tdj\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") " pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.619885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.619805    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:13.787892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.787867    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:44:13.789348 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:44:13.789319    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c4b90ab_e882_4124_b3df_65f815606e10.slice/crio-3f82b66ccb52c0d922bb759041d0bf36bb1e83bda4c15d98baf0bb40dfdc3642 WatchSource:0}: Error finding container 3f82b66ccb52c0d922bb759041d0bf36bb1e83bda4c15d98baf0bb40dfdc3642: Status 404 returned error can't find the container with id 3f82b66ccb52c0d922bb759041d0bf36bb1e83bda4c15d98baf0bb40dfdc3642
Nov 01 03:44:13.848159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.848133    4892 generic.go:356] "Generic (PLEG): container finished" podID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerID="33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3" exitCode=0
Nov 01 03:44:13.848289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.848202    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerDied","Data":"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"}
Nov 01 03:44:13.848654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.848636    4892 scope.go:122] "RemoveContainer" containerID="33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"
Nov 01 03:44:13.850130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.850108    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" event={"ID":"6c4b90ab-e882-4124-b3df-65f815606e10","Type":"ContainerStarted","Data":"3f82b66ccb52c0d922bb759041d0bf36bb1e83bda4c15d98baf0bb40dfdc3642"}
Nov 01 03:44:13.850790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.850754    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:13.850866 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.850850    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:13.850916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.850900    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:13.998609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:13.998579    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:44:14.017087 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:44:14.008698    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod140b948a_f5f0_4d16_b601_ea8f97b35cc8.slice/crio-10097f8b6a61fc8b5e2cd8f9cac12186d1d74b2eacf282db98fe8e94b6d7d528 WatchSource:0}: Error finding container 10097f8b6a61fc8b5e2cd8f9cac12186d1d74b2eacf282db98fe8e94b6d7d528: Status 404 returned error can't find the container with id 10097f8b6a61fc8b5e2cd8f9cac12186d1d74b2eacf282db98fe8e94b6d7d528
Nov 01 03:44:14.865318 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.865276    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" event={"ID":"6c4b90ab-e882-4124-b3df-65f815606e10","Type":"ContainerStarted","Data":"fcc33fc956008be14a1b18d115b73b24de29e8097c96f41cb550ffe44c5ff47d"}
Nov 01 03:44:14.865739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.865349    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:14.867276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.867241    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" event={"ID":"140b948a-f5f0-4d16-b601-ea8f97b35cc8","Type":"ContainerStarted","Data":"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"}
Nov 01 03:44:14.867415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.867300    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" event={"ID":"140b948a-f5f0-4d16-b601-ea8f97b35cc8","Type":"ContainerStarted","Data":"10097f8b6a61fc8b5e2cd8f9cac12186d1d74b2eacf282db98fe8e94b6d7d528"}
Nov 01 03:44:14.867415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.867329    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:14.869780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.869757    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerStarted","Data":"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"}
Nov 01 03:44:14.983355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:14.983307    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" podStartSLOduration=1.983292426 podStartE2EDuration="1.983292426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:44:14.98178199 +0000 UTC m=+2221.397681542" watchObservedRunningTime="2026-11-01 03:44:14.983292426 +0000 UTC m=+2221.399192080"
Nov 01 03:44:15.026375 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.026342    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:44:15.046147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.046113    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:15.046320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.046171    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:15.116534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.116431    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" podStartSLOduration=2.116416599 podStartE2EDuration="2.116416599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:44:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:44:15.111157908 +0000 UTC m=+2221.527057434" watchObservedRunningTime="2026-11-01 03:44:15.116416599 +0000 UTC m=+2221.532316129"
Nov 01 03:44:15.666314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.666118    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:44:15.666512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:15.666382    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" containerID="cri-o://068f29e332ece196b73aa912bd7d8da060418d1d1191b06f9f4dadbb34718cae" gracePeriod=30
Nov 01 03:44:16.036003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:16.034470    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:16.036003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:16.034547    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:17.032491 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:17.032451    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:17.032796 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:17.032765    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:18.034005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:18.026405    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:44:18.034005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:18.027447    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:18.034005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:18.027505    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:19.025405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:19.025368    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:19.025702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:19.025669    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:20.031092 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.031041    4892 patch_prober.go:29] interesting pod/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused" start-of-body=
Nov 01 03:44:20.036081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.036046    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.102:15021/healthz/ready\": dial tcp 10.134.0.102:15021: connect: connection refused"
Nov 01 03:44:20.588653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.588616    4892 patch_prober.go:29] interesting pod/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:44:20.588824 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.588680    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:44:20.909895 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.909864    4892 generic.go:356] "Generic (PLEG): container finished" podID="387d4451-4574-458f-8f1c-230507800e48" containerID="068f29e332ece196b73aa912bd7d8da060418d1d1191b06f9f4dadbb34718cae" exitCode=0
Nov 01 03:44:20.910070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.909950    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerDied","Data":"068f29e332ece196b73aa912bd7d8da060418d1d1191b06f9f4dadbb34718cae"}
Nov 01 03:44:20.910070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.910039    4892 scope.go:122] "RemoveContainer" containerID="cfc3e8364730614c75e403e6a4e37e15208071d9b8884ed46928ad0d75e760cf"
Nov 01 03:44:20.941402 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:20.941377    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:44:21.027750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.027720    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:44:21.028841 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.028812    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:44:21.066320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066288    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066346    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066405    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066429    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066493    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066535    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066590    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066612    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066629    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:44:21.066757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066724    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\") pod \"387d4451-4574-458f-8f1c-230507800e48\" (UID: \"387d4451-4574-458f-8f1c-230507800e48\") "
Nov 01 03:44:21.067283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066745    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:44:21.067283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066850    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:44:21.067283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.066868    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:44:21.067283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067267    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:44:21.067507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067358    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.067507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067377    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.067507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067395    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/387d4451-4574-458f-8f1c-230507800e48-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.067507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067409    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.067507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.067420    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.068813 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.068786    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:44:21.068919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.068905    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:44:21.069020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.069005    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "kube-api-access-pqvgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:44:21.069104 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.069089    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy" pod "387d4451-4574-458f-8f1c-230507800e48" (UID: "387d4451-4574-458f-8f1c-230507800e48"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:44:21.169278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.169191    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/387d4451-4574-458f-8f1c-230507800e48-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.169278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.169222    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqvgm\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-kube-api-access-pqvgm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.169278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.169237    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/387d4451-4574-458f-8f1c-230507800e48-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.169278 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.169250    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/387d4451-4574-458f-8f1c-230507800e48-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:44:21.918065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.918033    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"
Nov 01 03:44:21.918065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.918053    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2" event={"ID":"387d4451-4574-458f-8f1c-230507800e48","Type":"ContainerDied","Data":"c71e74c23dc29090183539881091db38e8a292f9638dddb1e02f25903d122c1a"}
Nov 01 03:44:21.918327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:21.918120    4892 scope.go:122] "RemoveContainer" containerID="068f29e332ece196b73aa912bd7d8da060418d1d1191b06f9f4dadbb34718cae"
Nov 01 03:44:22.218788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:22.218749    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:44:22.224111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:22.224073    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f2383c-openshift-default-65d58fc79f-9w7p2"]
Nov 01 03:44:23.701483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:23.701441    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="387d4451-4574-458f-8f1c-230507800e48" path="/var/lib/kubelet/pods/387d4451-4574-458f-8f1c-230507800e48/volumes"
Nov 01 03:44:25.881411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:25.881380    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:44:25.881794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:25.881450    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:44:27.616073 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:44:27.616037    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:44:55.217201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.217163    4892 generic.go:356] "Generic (PLEG): container finished" podID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerID="7812abc0e90c4a291674d18e199706075e808f22cf180f0629480f538d516b27" exitCode=0
Nov 01 03:44:55.217701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.217233    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerDied","Data":"7812abc0e90c4a291674d18e199706075e808f22cf180f0629480f538d516b27"}
Nov 01 03:44:55.217701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.217691    4892 scope.go:122] "RemoveContainer" containerID="7812abc0e90c4a291674d18e199706075e808f22cf180f0629480f538d516b27"
Nov 01 03:44:55.219254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219234    4892 generic.go:356] "Generic (PLEG): container finished" podID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerID="288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5" exitCode=0
Nov 01 03:44:55.219349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219286    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerDied","Data":"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"}
Nov 01 03:44:55.219422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219395    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:55.219471 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219463    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:55.219508 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219494    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:55.219573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.219561    4892 scope.go:122] "RemoveContainer" containerID="288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"
Nov 01 03:44:55.220893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.220867    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:55.220971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.220930    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:55.221047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:55.220970    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:44:56.276054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:56.273641    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerStarted","Data":"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"}
Nov 01 03:44:56.282753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:56.282717    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerStarted","Data":"b4a1397d3c8e839375547b179f947dcfd1bdaaf90e710483b5401d857b886383"}
Nov 01 03:44:56.691919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:56.691838    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:44:56.693081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:56.693050    4892 patch_prober.go:29] interesting pod/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 01 03:44:56.693206 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:56.693118    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 01 03:44:57.251455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:57.251420    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:44:57.251869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:57.251763    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:44:57.252079 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:57.252055    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:44:57.614858 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:44:57.614797    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:44:57.691517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:57.691485    4892 patch_prober.go:29] interesting pod/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 01 03:44:57.691753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:57.691731    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 01 03:44:58.154201 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:58.154093    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:44:58.253002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:58.252437    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:44:58.253002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:58.252501    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:44:58.691839 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:58.691800    4892 patch_prober.go:29] interesting pod/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 01 03:44:58.692285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:58.691853    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 01 03:44:59.250457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:59.250417    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:44:59.250762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:59.250727    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:44:59.692086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:59.692047    4892 patch_prober.go:29] interesting pod/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 01 03:44:59.692473 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:44:59.692102    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 01 03:45:00.212034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.211992    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:00.212598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212582    4892 cpu_manager.go:414] "containerMap: removing container" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.212648 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212600    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.212648 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212615    4892 cpu_manager.go:414] "containerMap: removing container" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.212648 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212620    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.212742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212732    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.212742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.212741    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="387d4451-4574-458f-8f1c-230507800e48" containerName="istio-proxy"
Nov 01 03:45:00.215180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.215161    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.256446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.255529    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:45:00.256446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.255585    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:45:00.274241 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.274200    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:45:00.277782 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.277739    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:00.281237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.281213    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.357006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.356050    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:00.374105 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.374069    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2qfj\" (UniqueName: \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.374544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.374519    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.374790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.374771    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.374956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.374938    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4v87\" (UniqueName: \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.377391 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.377366    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:00.379899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.379858    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.380968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.380950    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:00.381242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.381206    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.384539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.384509    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:00.386382 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.386360    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.388259 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.388237    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:00.388367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.388342    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.391100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.391076    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:00.395474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.395457    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.415868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.415818    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:00.420698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.420651    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:00.485061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.484765    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4v87\" (UniqueName: \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.485061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.484825    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.485061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.484864    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8ps7\" (UniqueName: \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.485061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.484898    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.485061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.484941    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.489271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.489202    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxrs2\" (UniqueName: \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.489578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.489529    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.489734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.489692    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvr7b\" (UniqueName: \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.489898    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.489944    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2qfj\" (UniqueName: \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.490020    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjvgf\" (UniqueName: \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.490154    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcfpx\" (UniqueName: \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.490243    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.490379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.490298    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.491027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.491003    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.491116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.491053    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:00.493498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.493471    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.499455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.499381    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:00.508428 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.508398    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:00.513615 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.513560    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:00.577124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.577092    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4v87\" (UniqueName: \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\") pod \"maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:00.581040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.579049    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2qfj\" (UniqueName: \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\") pod \"maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.592288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.592255    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.592557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.592538    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8ps7\" (UniqueName: \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.592695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.592677    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.592841 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.592825    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.593046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593026    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxrs2\" (UniqueName: \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.593208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593190    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.593328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593312    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvr7b\" (UniqueName: \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.593436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593421    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.593623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593607    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjvgf\" (UniqueName: \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.593760 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593745    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcfpx\" (UniqueName: \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.594235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593853    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.594235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.593766    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.594477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.594451    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.594544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.594458    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.594617 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.594597    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.595209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.595189    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:00.670004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.669886    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8ps7\" (UniqueName: \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\") pod \"maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.682032 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.681916    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcfpx\" (UniqueName: \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\") pod \"maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.682908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.682878    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvr7b\" (UniqueName: \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\") pod \"maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.683108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.683084    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxrs2\" (UniqueName: \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\") pod \"maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.685833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.685809    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjvgf\" (UniqueName: \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\") pod \"maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.692708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.692677    4892 patch_prober.go:29] interesting pod/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Nov 01 03:45:00.693202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.692740    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Nov 01 03:45:00.704654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.704618    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:00.786811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.786101    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:00.802990 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.802765    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:00.814251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.814206    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:00.892168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.851947    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:00.892168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:00.892055    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:01.291217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.290820    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:01.331022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.308169    4892 patch_prober.go:29] interesting pod/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Nov 01 03:45:01.331022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.308230    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Nov 01 03:45:01.444335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.444291    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:01.559701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.559673    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:01.631041 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:01.631004    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d762540_b099_47d2_ad61_4ffa8e589ac7.slice/crio-7b8ab673f8b4cdec951807f2116ddef9fbfd3be012f9b4119f75ff961cdfd147 WatchSource:0}: Error finding container 7b8ab673f8b4cdec951807f2116ddef9fbfd3be012f9b4119f75ff961cdfd147: Status 404 returned error can't find the container with id 7b8ab673f8b4cdec951807f2116ddef9fbfd3be012f9b4119f75ff961cdfd147
Nov 01 03:45:01.710292 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.710263    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:45:01.711642 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.711624    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:45:01.718667 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.718642    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:01.720346 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:01.720318    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2bb051a_7aa2_4367_9137_22d0db840c71.slice/crio-fd422f5ed7b1484d8b5550d4f4ff53d291fbde0f3a08a9c8e405c1c443b5a1ea WatchSource:0}: Error finding container fd422f5ed7b1484d8b5550d4f4ff53d291fbde0f3a08a9c8e405c1c443b5a1ea: Status 404 returned error can't find the container with id fd422f5ed7b1484d8b5550d4f4ff53d291fbde0f3a08a9c8e405c1c443b5a1ea
Nov 01 03:45:01.746189 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.746161    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:01.824264 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:01.824220    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5f369eb_640b_42d2_8c99_41f588351733.slice/crio-c54c664b5157b724a7552574f1cc2abfc41d0370cad1c3139451904b525f58d9 WatchSource:0}: Error finding container c54c664b5157b724a7552574f1cc2abfc41d0370cad1c3139451904b525f58d9: Status 404 returned error can't find the container with id c54c664b5157b724a7552574f1cc2abfc41d0370cad1c3139451904b525f58d9
Nov 01 03:45:01.861329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.861294    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:01.864553 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:01.864523    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3176562_080f_41c2_894c_6ed32fc86a07.slice/crio-77cd6a01275d9453d4971742052fb105058e6f1d7712db44336362954f622930 WatchSource:0}: Error finding container 77cd6a01275d9453d4971742052fb105058e6f1d7712db44336362954f622930: Status 404 returned error can't find the container with id 77cd6a01275d9453d4971742052fb105058e6f1d7712db44336362954f622930
Nov 01 03:45:01.868686 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:01.868654    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:01.874153 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:01.873942    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c47d360_0612_4329_81e2_f85b667ef6dd.slice/crio-e4279592892c40d9a2fb65cae7cd30f2060e93dc16926920b560d65dde6ce5af WatchSource:0}: Error finding container e4279592892c40d9a2fb65cae7cd30f2060e93dc16926920b560d65dde6ce5af: Status 404 returned error can't find the container with id e4279592892c40d9a2fb65cae7cd30f2060e93dc16926920b560d65dde6ce5af
Nov 01 03:45:02.255327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.255295    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:45:02.255582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.255563    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:45:02.256477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.256456    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:45:02.330942 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.330901    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerStarted","Data":"5cf170a504eb5e14b0d590bf1c3dab74c91c82baf8c8f43df687cda983365d87"}
Nov 01 03:45:02.331125 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.330966    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerStarted","Data":"77cd6a01275d9453d4971742052fb105058e6f1d7712db44336362954f622930"}
Nov 01 03:45:02.332485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.332462    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerStarted","Data":"f4c9b19c981e7d212b90fcd488f3a2cf995243bb9ec35a9ca90e4d3afee027c9"}
Nov 01 03:45:02.332603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.332500    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerStarted","Data":"7b8ab673f8b4cdec951807f2116ddef9fbfd3be012f9b4119f75ff961cdfd147"}
Nov 01 03:45:02.333861 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.333840    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerStarted","Data":"80f3aad6b0ccd72a73140bd57b3ddcb5f49f62913881fc3cde2fcd3c36774e68"}
Nov 01 03:45:02.333920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.333885    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerStarted","Data":"fd422f5ed7b1484d8b5550d4f4ff53d291fbde0f3a08a9c8e405c1c443b5a1ea"}
Nov 01 03:45:02.335258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.335239    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerStarted","Data":"667e45baafbdf65d6b49c0b2a8b131f213302ba1ff535972b6e36c5df6c2951d"}
Nov 01 03:45:02.335344 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.335277    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerStarted","Data":"1f23190d36f835ecb2f140afa2011fd136b86266c6dbd6357e18fe0da15a00d3"}
Nov 01 03:45:02.336885 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.336862    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerStarted","Data":"f1f9059300147d1e9c73d46ba9f9e104ab3875475e87a9503be59cc5e289c0ef"}
Nov 01 03:45:02.337016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.336905    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerStarted","Data":"d5844b7b9b72746507fed181dd356a4f13bb821a89410832614283a3957ac920"}
Nov 01 03:45:02.338083 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.338063    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerStarted","Data":"e4279592892c40d9a2fb65cae7cd30f2060e93dc16926920b560d65dde6ce5af"}
Nov 01 03:45:02.339513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.339489    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerStarted","Data":"e02a4a079bbceb5298d2d5533a589591c9a081f78112f4a272d483e95c2ccb79"}
Nov 01 03:45:02.339589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.339528    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerStarted","Data":"c54c664b5157b724a7552574f1cc2abfc41d0370cad1c3139451904b525f58d9"}
Nov 01 03:45:02.508625 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.508565    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" podStartSLOduration=2.50854605 podStartE2EDuration="2.50854605s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:02.505792381 +0000 UTC m=+2268.921691912" watchObservedRunningTime="2026-11-01 03:45:02.50854605 +0000 UTC m=+2268.924445580"
Nov 01 03:45:02.624704 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.624627    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" podStartSLOduration=2.6246042210000002 podStartE2EDuration="2.624604221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:02.621502075 +0000 UTC m=+2269.037401599" watchObservedRunningTime="2026-11-01 03:45:02.624604221 +0000 UTC m=+2269.040503815"
Nov 01 03:45:02.751561 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.751504    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" podStartSLOduration=2.751489381 podStartE2EDuration="2.751489381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:02.74931674 +0000 UTC m=+2269.165216269" watchObservedRunningTime="2026-11-01 03:45:02.751489381 +0000 UTC m=+2269.167388910"
Nov 01 03:45:02.868849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.868794    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" podStartSLOduration=2.868778966 podStartE2EDuration="2.868778966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:02.865349215 +0000 UTC m=+2269.281248743" watchObservedRunningTime="2026-11-01 03:45:02.868778966 +0000 UTC m=+2269.284678494"
Nov 01 03:45:02.989407 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:02.989349    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" podStartSLOduration=2.989334258 podStartE2EDuration="2.989334258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:02.989092654 +0000 UTC m=+2269.404992177" watchObservedRunningTime="2026-11-01 03:45:02.989334258 +0000 UTC m=+2269.405233786"
Nov 01 03:45:03.246411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.246302    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" podStartSLOduration=3.246284557 podStartE2EDuration="3.246284557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:03.243685928 +0000 UTC m=+2269.659585450" watchObservedRunningTime="2026-11-01 03:45:03.246284557 +0000 UTC m=+2269.662184082"
Nov 01 03:45:03.348739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.348702    4892 generic.go:356] "Generic (PLEG): container finished" podID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerID="80f3aad6b0ccd72a73140bd57b3ddcb5f49f62913881fc3cde2fcd3c36774e68" exitCode=60
Nov 01 03:45:03.348914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.348798    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerDied","Data":"80f3aad6b0ccd72a73140bd57b3ddcb5f49f62913881fc3cde2fcd3c36774e68"}
Nov 01 03:45:03.349193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.349165    4892 scope.go:122] "RemoveContainer" containerID="80f3aad6b0ccd72a73140bd57b3ddcb5f49f62913881fc3cde2fcd3c36774e68"
Nov 01 03:45:03.350683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.350654    4892 generic.go:356] "Generic (PLEG): container finished" podID="73c59c5b-d898-4d57-a84c-f72f46719315" containerID="667e45baafbdf65d6b49c0b2a8b131f213302ba1ff535972b6e36c5df6c2951d" exitCode=60
Nov 01 03:45:03.350780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.350760    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerDied","Data":"667e45baafbdf65d6b49c0b2a8b131f213302ba1ff535972b6e36c5df6c2951d"}
Nov 01 03:45:03.351033 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.351014    4892 scope.go:122] "RemoveContainer" containerID="667e45baafbdf65d6b49c0b2a8b131f213302ba1ff535972b6e36c5df6c2951d"
Nov 01 03:45:03.354675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.353525    4892 generic.go:356] "Generic (PLEG): container finished" podID="389a0c21-02b9-4815-a797-60800f3fbc51" containerID="f1f9059300147d1e9c73d46ba9f9e104ab3875475e87a9503be59cc5e289c0ef" exitCode=60
Nov 01 03:45:03.354675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.353579    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerDied","Data":"f1f9059300147d1e9c73d46ba9f9e104ab3875475e87a9503be59cc5e289c0ef"}
Nov 01 03:45:03.354675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.354161    4892 scope.go:122] "RemoveContainer" containerID="f1f9059300147d1e9c73d46ba9f9e104ab3875475e87a9503be59cc5e289c0ef"
Nov 01 03:45:03.355626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.355603    4892 generic.go:356] "Generic (PLEG): container finished" podID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerID="c3ce8ee8faf5b8d37d5ff842c7999ea2e66394f29650bd76038ea348471b3f68" exitCode=60
Nov 01 03:45:03.355715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.355684    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerDied","Data":"c3ce8ee8faf5b8d37d5ff842c7999ea2e66394f29650bd76038ea348471b3f68"}
Nov 01 03:45:03.355901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.355884    4892 scope.go:122] "RemoveContainer" containerID="c3ce8ee8faf5b8d37d5ff842c7999ea2e66394f29650bd76038ea348471b3f68"
Nov 01 03:45:03.357401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.357378    4892 generic.go:356] "Generic (PLEG): container finished" podID="d5f369eb-640b-42d2-8c99-41f588351733" containerID="e02a4a079bbceb5298d2d5533a589591c9a081f78112f4a272d483e95c2ccb79" exitCode=60
Nov 01 03:45:03.357503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.357465    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerDied","Data":"e02a4a079bbceb5298d2d5533a589591c9a081f78112f4a272d483e95c2ccb79"}
Nov 01 03:45:03.357685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.357668    4892 scope.go:122] "RemoveContainer" containerID="e02a4a079bbceb5298d2d5533a589591c9a081f78112f4a272d483e95c2ccb79"
Nov 01 03:45:03.359518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.359494    4892 generic.go:356] "Generic (PLEG): container finished" podID="a3176562-080f-41c2-894c-6ed32fc86a07" containerID="5cf170a504eb5e14b0d590bf1c3dab74c91c82baf8c8f43df687cda983365d87" exitCode=60
Nov 01 03:45:03.359879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.359595    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerDied","Data":"5cf170a504eb5e14b0d590bf1c3dab74c91c82baf8c8f43df687cda983365d87"}
Nov 01 03:45:03.360018 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.360001    4892 scope.go:122] "RemoveContainer" containerID="5cf170a504eb5e14b0d590bf1c3dab74c91c82baf8c8f43df687cda983365d87"
Nov 01 03:45:03.362656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.362638    4892 generic.go:356] "Generic (PLEG): container finished" podID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerID="f4c9b19c981e7d212b90fcd488f3a2cf995243bb9ec35a9ca90e4d3afee027c9" exitCode=60
Nov 01 03:45:03.362750 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.362727    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerDied","Data":"f4c9b19c981e7d212b90fcd488f3a2cf995243bb9ec35a9ca90e4d3afee027c9"}
Nov 01 03:45:03.362998 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:03.362969    4892 scope.go:122] "RemoveContainer" containerID="f4c9b19c981e7d212b90fcd488f3a2cf995243bb9ec35a9ca90e4d3afee027c9"
Nov 01 03:45:04.370992 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.370939    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerStarted","Data":"44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"}
Nov 01 03:45:04.372879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.372854    4892 generic.go:356] "Generic (PLEG): container finished" podID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerID="f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a" exitCode=60
Nov 01 03:45:04.373019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.372933    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerDied","Data":"f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a"}
Nov 01 03:45:04.373081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.373030    4892 scope.go:122] "RemoveContainer" containerID="80f3aad6b0ccd72a73140bd57b3ddcb5f49f62913881fc3cde2fcd3c36774e68"
Nov 01 03:45:04.373192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.373172    4892 scope.go:122] "RemoveContainer" containerID="f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a"
Nov 01 03:45:04.373497 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:04.373470    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd_odh-ai-gateway-infra(d2bb051a-7aa2-4367-9137-22d0db840c71)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71"
Nov 01 03:45:04.374854 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.374833    4892 generic.go:356] "Generic (PLEG): container finished" podID="73c59c5b-d898-4d57-a84c-f72f46719315" containerID="42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e" exitCode=60
Nov 01 03:45:04.374943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.374898    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerDied","Data":"42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e"}
Nov 01 03:45:04.375162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.375146    4892 scope.go:122] "RemoveContainer" containerID="42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e"
Nov 01 03:45:04.375405 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:04.375388    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77_odh-ai-gateway-infra(73c59c5b-d898-4d57-a84c-f72f46719315)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" podUID="73c59c5b-d898-4d57-a84c-f72f46719315"
Nov 01 03:45:04.376803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.376776    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerStarted","Data":"636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"}
Nov 01 03:45:04.378658 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.378633    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerStarted","Data":"ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"}
Nov 01 03:45:04.380427 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.380407    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerStarted","Data":"fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"}
Nov 01 03:45:04.382074 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.382054    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerStarted","Data":"192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3"}
Nov 01 03:45:04.403387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:04.403363    4892 scope.go:122] "RemoveContainer" containerID="667e45baafbdf65d6b49c0b2a8b131f213302ba1ff535972b6e36c5df6c2951d"
Nov 01 03:45:05.092038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.091964    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" podStartSLOduration=5.091947702 podStartE2EDuration="5.091947702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:05.089375107 +0000 UTC m=+2271.505274648" watchObservedRunningTime="2026-11-01 03:45:05.091947702 +0000 UTC m=+2271.507847233"
Nov 01 03:45:05.391547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.391456    4892 generic.go:356] "Generic (PLEG): container finished" podID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerID="44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594" exitCode=60
Nov 01 03:45:05.391547 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.391533    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerDied","Data":"44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"}
Nov 01 03:45:05.392139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.391588    4892 scope.go:122] "RemoveContainer" containerID="f4c9b19c981e7d212b90fcd488f3a2cf995243bb9ec35a9ca90e4d3afee027c9"
Nov 01 03:45:05.392139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.391911    4892 scope.go:122] "RemoveContainer" containerID="44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"
Nov 01 03:45:05.392309 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.392289    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx_odh-ai-gateway-infra(0d762540-b099-47d2-ad61-4ffa8e589ac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7"
Nov 01 03:45:05.393804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.393781    4892 scope.go:122] "RemoveContainer" containerID="f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a"
Nov 01 03:45:05.394101 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.394078    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd_odh-ai-gateway-infra(d2bb051a-7aa2-4367-9137-22d0db840c71)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71"
Nov 01 03:45:05.395414 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.395398    4892 scope.go:122] "RemoveContainer" containerID="42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e"
Nov 01 03:45:05.395662 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.395641    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77_odh-ai-gateway-infra(73c59c5b-d898-4d57-a84c-f72f46719315)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" podUID="73c59c5b-d898-4d57-a84c-f72f46719315"
Nov 01 03:45:05.397260 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.397239    4892 generic.go:356] "Generic (PLEG): container finished" podID="389a0c21-02b9-4815-a797-60800f3fbc51" containerID="636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a" exitCode=60
Nov 01 03:45:05.397340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.397318    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerDied","Data":"636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"}
Nov 01 03:45:05.397519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.397502    4892 scope.go:122] "RemoveContainer" containerID="636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"
Nov 01 03:45:05.397776 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.397754    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg_odh-ai-gateway-infra(389a0c21-02b9-4815-a797-60800f3fbc51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" podUID="389a0c21-02b9-4815-a797-60800f3fbc51"
Nov 01 03:45:05.399239 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.399218    4892 generic.go:356] "Generic (PLEG): container finished" podID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerID="ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a" exitCode=60
Nov 01 03:45:05.399340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.399303    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerDied","Data":"ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"}
Nov 01 03:45:05.399479 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.399465    4892 scope.go:122] "RemoveContainer" containerID="ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"
Nov 01 03:45:05.399711 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.399689    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr_odh-ai-gateway-infra(4c47d360-0612-4329-81e2-f85b667ef6dd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd"
Nov 01 03:45:05.401124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.401104    4892 generic.go:356] "Generic (PLEG): container finished" podID="d5f369eb-640b-42d2-8c99-41f588351733" containerID="fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd" exitCode=60
Nov 01 03:45:05.401222 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.401189    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerDied","Data":"fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"}
Nov 01 03:45:05.401379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.401364    4892 scope.go:122] "RemoveContainer" containerID="fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"
Nov 01 03:45:05.401584 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.401564    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg_odh-ai-gateway-infra(d5f369eb-640b-42d2-8c99-41f588351733)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" podUID="d5f369eb-640b-42d2-8c99-41f588351733"
Nov 01 03:45:05.403100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.403078    4892 generic.go:356] "Generic (PLEG): container finished" podID="a3176562-080f-41c2-894c-6ed32fc86a07" containerID="192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3" exitCode=60
Nov 01 03:45:05.403182 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.403139    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerDied","Data":"192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3"}
Nov 01 03:45:05.403291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.403278    4892 scope.go:122] "RemoveContainer" containerID="192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3"
Nov 01 03:45:05.403473 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:05.403455    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh_odh-ai-gateway-infra(a3176562-080f-41c2-894c-6ed32fc86a07)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" podUID="a3176562-080f-41c2-894c-6ed32fc86a07"
Nov 01 03:45:05.404432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.404418    4892 scope.go:122] "RemoveContainer" containerID="f1f9059300147d1e9c73d46ba9f9e104ab3875475e87a9503be59cc5e289c0ef"
Nov 01 03:45:05.415359 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.415337    4892 scope.go:122] "RemoveContainer" containerID="c3ce8ee8faf5b8d37d5ff842c7999ea2e66394f29650bd76038ea348471b3f68"
Nov 01 03:45:05.427034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.427014    4892 scope.go:122] "RemoveContainer" containerID="e02a4a079bbceb5298d2d5533a589591c9a081f78112f4a272d483e95c2ccb79"
Nov 01 03:45:05.437544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:05.437523    4892 scope.go:122] "RemoveContainer" containerID="5cf170a504eb5e14b0d590bf1c3dab74c91c82baf8c8f43df687cda983365d87"
Nov 01 03:45:06.413003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:06.412948    4892 scope.go:122] "RemoveContainer" containerID="192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3"
Nov 01 03:45:06.413454 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:06.413337    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh_odh-ai-gateway-infra(a3176562-080f-41c2-894c-6ed32fc86a07)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" podUID="a3176562-080f-41c2-894c-6ed32fc86a07"
Nov 01 03:45:06.414557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:06.414539    4892 scope.go:122] "RemoveContainer" containerID="44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"
Nov 01 03:45:06.414844 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:06.414818    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx_odh-ai-gateway-infra(0d762540-b099-47d2-ad61-4ffa8e589ac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7"
Nov 01 03:45:06.416197 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:06.416178    4892 scope.go:122] "RemoveContainer" containerID="636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"
Nov 01 03:45:06.416437 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:06.416407    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg_odh-ai-gateway-infra(389a0c21-02b9-4815-a797-60800f3fbc51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" podUID="389a0c21-02b9-4815-a797-60800f3fbc51"
Nov 01 03:45:06.417727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:06.417708    4892 scope.go:122] "RemoveContainer" containerID="ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"
Nov 01 03:45:06.417923 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:06.417902    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr_odh-ai-gateway-infra(4c47d360-0612-4329-81e2-f85b667ef6dd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd"
Nov 01 03:45:06.419265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:06.419249    4892 scope.go:122] "RemoveContainer" containerID="fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"
Nov 01 03:45:06.419435 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:06.419418    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg_odh-ai-gateway-infra(d5f369eb-640b-42d2-8c99-41f588351733)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" podUID="d5f369eb-640b-42d2-8c99-41f588351733"
Nov 01 03:45:08.664315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:08.664280    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:08.953323 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:08.941789    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:09.038203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.038156    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\") pod \"a3176562-080f-41c2-894c-6ed32fc86a07\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") "
Nov 01 03:45:09.038421 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.038406    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\" (UniqueName: \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\") pod \"a3176562-080f-41c2-894c-6ed32fc86a07\" (UID: \"a3176562-080f-41c2-894c-6ed32fc86a07\") "
Nov 01 03:45:09.038605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.038568    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca" pod "a3176562-080f-41c2-894c-6ed32fc86a07" (UID: "a3176562-080f-41c2-894c-6ed32fc86a07"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:09.038801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.038785    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3176562-080f-41c2-894c-6ed32fc86a07-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:09.041165 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.040823    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87" pod "a3176562-080f-41c2-894c-6ed32fc86a07" (UID: "a3176562-080f-41c2-894c-6ed32fc86a07"). InnerVolumeSpecName "kube-api-access-s4v87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:09.140724 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.140688    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4v87\" (UniqueName: \"kubernetes.io/projected/a3176562-080f-41c2-894c-6ed32fc86a07-kube-api-access-s4v87\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:09.455209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.452688    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh" event={"ID":"a3176562-080f-41c2-894c-6ed32fc86a07","Type":"ContainerDied","Data":"77cd6a01275d9453d4971742052fb105058e6f1d7712db44336362954f622930"}
Nov 01 03:45:09.455209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.452806    4892 scope.go:122] "RemoveContainer" containerID="192046202ccb0833fcaabd8831182a81038fcee53d7dbf356a2d61cb530248d3"
Nov 01 03:45:09.455209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.452957    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"
Nov 01 03:45:09.621959 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.621920    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:09.633971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.633940    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w4-e8c789-29891745-rczkh"]
Nov 01 03:45:09.703802 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:09.703770    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" path="/var/lib/kubelet/pods/a3176562-080f-41c2-894c-6ed32fc86a07/volumes"
Nov 01 03:45:15.694920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:15.694888    4892 scope.go:122] "RemoveContainer" containerID="44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"
Nov 01 03:45:15.695435 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:15.695059    4892 scope.go:122] "RemoveContainer" containerID="636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"
Nov 01 03:45:16.507729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:16.507685    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerStarted","Data":"62759c98efdcc82bb317f20d76f17bd7c865400351d1beb6d8bbf8564672ebf5"}
Nov 01 03:45:16.510301 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:16.510265    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerStarted","Data":"c438154bd98d6feb02cc3840acf215b3b166a2ca46f5d82f534724daf5e2ad26"}
Nov 01 03:45:17.520609 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.520567    4892 generic.go:356] "Generic (PLEG): container finished" podID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerID="62759c98efdcc82bb317f20d76f17bd7c865400351d1beb6d8bbf8564672ebf5" exitCode=60
Nov 01 03:45:17.521162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.520612    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerDied","Data":"62759c98efdcc82bb317f20d76f17bd7c865400351d1beb6d8bbf8564672ebf5"}
Nov 01 03:45:17.521162 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.520700    4892 scope.go:122] "RemoveContainer" containerID="44d2bb83f054c58068ed9d3931bfd1689a2ed983d47a2b95d95893d79c2c8594"
Nov 01 03:45:17.521285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.521206    4892 scope.go:122] "RemoveContainer" containerID="62759c98efdcc82bb317f20d76f17bd7c865400351d1beb6d8bbf8564672ebf5"
Nov 01 03:45:17.521564 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:17.521539    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx_odh-ai-gateway-infra(0d762540-b099-47d2-ad61-4ffa8e589ac7)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7"
Nov 01 03:45:17.523073 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.523054    4892 generic.go:356] "Generic (PLEG): container finished" podID="389a0c21-02b9-4815-a797-60800f3fbc51" containerID="c438154bd98d6feb02cc3840acf215b3b166a2ca46f5d82f534724daf5e2ad26" exitCode=60
Nov 01 03:45:17.523193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.523088    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerDied","Data":"c438154bd98d6feb02cc3840acf215b3b166a2ca46f5d82f534724daf5e2ad26"}
Nov 01 03:45:17.523456 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.523439    4892 scope.go:122] "RemoveContainer" containerID="c438154bd98d6feb02cc3840acf215b3b166a2ca46f5d82f534724daf5e2ad26"
Nov 01 03:45:17.523748 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:17.523728    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg_odh-ai-gateway-infra(389a0c21-02b9-4815-a797-60800f3fbc51)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" podUID="389a0c21-02b9-4815-a797-60800f3fbc51"
Nov 01 03:45:17.532361 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.532338    4892 scope.go:122] "RemoveContainer" containerID="636103978a0abd737daab2d59c6093aaf935d92b9a5e7c1a615a871b2ac6350a"
Nov 01 03:45:17.654407 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.654372    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:17.694766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.694720    4892 scope.go:122] "RemoveContainer" containerID="f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a"
Nov 01 03:45:17.694942 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.694893    4892 scope.go:122] "RemoveContainer" containerID="42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e"
Nov 01 03:45:17.695059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.695041    4892 scope.go:122] "RemoveContainer" containerID="fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"
Nov 01 03:45:17.695120 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.695102    4892 scope.go:122] "RemoveContainer" containerID="ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"
Nov 01 03:45:17.806077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:17.805992    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:18.535095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.535060    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerStarted","Data":"457e340534ff476020e75336692ac6463d7876836a07961729001fa9713f1a2b"}
Nov 01 03:45:18.537255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.537225    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerStarted","Data":"4fff42d6a062e980f3d2115e4d7bf7f25851f045cd4c142312a35826ea082393"}
Nov 01 03:45:18.540969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.540942    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerStarted","Data":"3b407e1f77cfb8e1762b941943ba1bb3f6e3c29a5511645d11caa289dd2b9136"}
Nov 01 03:45:18.543052 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.543030    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerStarted","Data":"aa759327a47f6f467a5f79bfd3e9ce3474582f423a0e4028b5108e56b823c192"}
Nov 01 03:45:18.737333 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.737237    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:18.787646 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.787614    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\" (UniqueName: \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\") pod \"0d762540-b099-47d2-ad61-4ffa8e589ac7\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") "
Nov 01 03:45:18.787801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.787738    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\") pod \"0d762540-b099-47d2-ad61-4ffa8e589ac7\" (UID: \"0d762540-b099-47d2-ad61-4ffa8e589ac7\") "
Nov 01 03:45:18.788288 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.788263    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca" pod "0d762540-b099-47d2-ad61-4ffa8e589ac7" (UID: "0d762540-b099-47d2-ad61-4ffa8e589ac7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:18.789864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.789841    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b" pod "0d762540-b099-47d2-ad61-4ffa8e589ac7" (UID: "0d762540-b099-47d2-ad61-4ffa8e589ac7"). InnerVolumeSpecName "kube-api-access-fvr7b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:18.806270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.806245    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:18.888997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.888955    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\" (UniqueName: \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\") pod \"389a0c21-02b9-4815-a797-60800f3fbc51\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") "
Nov 01 03:45:18.889151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.889056    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\") pod \"389a0c21-02b9-4815-a797-60800f3fbc51\" (UID: \"389a0c21-02b9-4815-a797-60800f3fbc51\") "
Nov 01 03:45:18.889416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.889383    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca" pod "389a0c21-02b9-4815-a797-60800f3fbc51" (UID: "389a0c21-02b9-4815-a797-60800f3fbc51"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:18.889607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.889443    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvr7b\" (UniqueName: \"kubernetes.io/projected/0d762540-b099-47d2-ad61-4ffa8e589ac7-kube-api-access-fvr7b\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:18.889607 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.889457    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d762540-b099-47d2-ad61-4ffa8e589ac7-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:18.891147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.891123    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx" pod "389a0c21-02b9-4815-a797-60800f3fbc51" (UID: "389a0c21-02b9-4815-a797-60800f3fbc51"). InnerVolumeSpecName "kube-api-access-bcfpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:18.993031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.992991    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcfpx\" (UniqueName: \"kubernetes.io/projected/389a0c21-02b9-4815-a797-60800f3fbc51-kube-api-access-bcfpx\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:18.993031 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:18.993027    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/389a0c21-02b9-4815-a797-60800f3fbc51-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:19.550256 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.550225    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"
Nov 01 03:45:19.550736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.550255    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx" event={"ID":"0d762540-b099-47d2-ad61-4ffa8e589ac7","Type":"ContainerDied","Data":"7b8ab673f8b4cdec951807f2116ddef9fbfd3be012f9b4119f75ff961cdfd147"}
Nov 01 03:45:19.550736 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.550323    4892 scope.go:122] "RemoveContainer" containerID="62759c98efdcc82bb317f20d76f17bd7c865400351d1beb6d8bbf8564672ebf5"
Nov 01 03:45:19.552450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.552427    4892 generic.go:356] "Generic (PLEG): container finished" podID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerID="457e340534ff476020e75336692ac6463d7876836a07961729001fa9713f1a2b" exitCode=60
Nov 01 03:45:19.552553 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.552499    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerDied","Data":"457e340534ff476020e75336692ac6463d7876836a07961729001fa9713f1a2b"}
Nov 01 03:45:19.552933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.552916    4892 scope.go:122] "RemoveContainer" containerID="457e340534ff476020e75336692ac6463d7876836a07961729001fa9713f1a2b"
Nov 01 03:45:19.553293 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:19.553269    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd_odh-ai-gateway-infra(d2bb051a-7aa2-4367-9137-22d0db840c71)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71"
Nov 01 03:45:19.554649 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.554628    4892 generic.go:356] "Generic (PLEG): container finished" podID="73c59c5b-d898-4d57-a84c-f72f46719315" containerID="4fff42d6a062e980f3d2115e4d7bf7f25851f045cd4c142312a35826ea082393" exitCode=60
Nov 01 03:45:19.554732 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.554702    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerDied","Data":"4fff42d6a062e980f3d2115e4d7bf7f25851f045cd4c142312a35826ea082393"}
Nov 01 03:45:19.555136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.555116    4892 scope.go:122] "RemoveContainer" containerID="4fff42d6a062e980f3d2115e4d7bf7f25851f045cd4c142312a35826ea082393"
Nov 01 03:45:19.555435 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:19.555408    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77_odh-ai-gateway-infra(73c59c5b-d898-4d57-a84c-f72f46719315)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" podUID="73c59c5b-d898-4d57-a84c-f72f46719315"
Nov 01 03:45:19.556163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.556144    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"
Nov 01 03:45:19.556254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.556173    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg" event={"ID":"389a0c21-02b9-4815-a797-60800f3fbc51","Type":"ContainerDied","Data":"d5844b7b9b72746507fed181dd356a4f13bb821a89410832614283a3957ac920"}
Nov 01 03:45:19.558286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.558267    4892 generic.go:356] "Generic (PLEG): container finished" podID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerID="3b407e1f77cfb8e1762b941943ba1bb3f6e3c29a5511645d11caa289dd2b9136" exitCode=60
Nov 01 03:45:19.558383 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.558332    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerDied","Data":"3b407e1f77cfb8e1762b941943ba1bb3f6e3c29a5511645d11caa289dd2b9136"}
Nov 01 03:45:19.558685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.558665    4892 scope.go:122] "RemoveContainer" containerID="3b407e1f77cfb8e1762b941943ba1bb3f6e3c29a5511645d11caa289dd2b9136"
Nov 01 03:45:19.558924 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:19.558903    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr_odh-ai-gateway-infra(4c47d360-0612-4329-81e2-f85b667ef6dd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd"
Nov 01 03:45:19.560448 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.560425    4892 generic.go:356] "Generic (PLEG): container finished" podID="d5f369eb-640b-42d2-8c99-41f588351733" containerID="aa759327a47f6f467a5f79bfd3e9ce3474582f423a0e4028b5108e56b823c192" exitCode=60
Nov 01 03:45:19.560545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.560457    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerDied","Data":"aa759327a47f6f467a5f79bfd3e9ce3474582f423a0e4028b5108e56b823c192"}
Nov 01 03:45:19.560936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.560915    4892 scope.go:122] "RemoveContainer" containerID="aa759327a47f6f467a5f79bfd3e9ce3474582f423a0e4028b5108e56b823c192"
Nov 01 03:45:19.561243 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:19.561217    4892 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg_odh-ai-gateway-infra(d5f369eb-640b-42d2-8c99-41f588351733)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" podUID="d5f369eb-640b-42d2-8c99-41f588351733"
Nov 01 03:45:19.562509 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.562493    4892 scope.go:122] "RemoveContainer" containerID="f5a002d6d233f4d45cb6cb761d248a8ca6c4759229d2500849b33bc2a5fea37a"
Nov 01 03:45:19.575351 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.575329    4892 scope.go:122] "RemoveContainer" containerID="42d29d2eed3747b7b854252aca40ac2e8fcee003440352574b972c4da968400e"
Nov 01 03:45:19.588436 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.588415    4892 scope.go:122] "RemoveContainer" containerID="c438154bd98d6feb02cc3840acf215b3b166a2ca46f5d82f534724daf5e2ad26"
Nov 01 03:45:19.598283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.598261    4892 scope.go:122] "RemoveContainer" containerID="ace7f78026fbab237540880be6fe5a9e1920437bc2d7f86a5240a2d56e00908a"
Nov 01 03:45:19.612015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.611991    4892 scope.go:122] "RemoveContainer" containerID="fb11ef03fb96b04c63587e022c0e39e008dc99f338f2bc08c70f06b18dff55cd"
Nov 01 03:45:19.778721 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.778685    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:19.951634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.951543    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:19.960120 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:19.960087    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-b12199-29891745-fsbbx"]
Nov 01 03:45:20.015389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.014918    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:20.072406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.072365    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:20.078735 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.078702    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-5fd6cf-29891745-gq5hg"]
Nov 01 03:45:20.153216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.153178    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:20.267097 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.267062    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:20.818336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.818245    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:20.870093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.870063    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:20.873705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.873677    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:20.877357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.877331    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:20.925530 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.925499    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\" (UniqueName: \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\") pod \"d5f369eb-640b-42d2-8c99-41f588351733\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") "
Nov 01 03:45:20.925723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.925582    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\") pod \"d2bb051a-7aa2-4367-9137-22d0db840c71\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.926167    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca" pod "d2bb051a-7aa2-4367-9137-22d0db840c71" (UID: "d2bb051a-7aa2-4367-9137-22d0db840c71"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.926632    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\" (UniqueName: \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\") pod \"73c59c5b-d898-4d57-a84c-f72f46719315\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.926708    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\") pod \"d5f369eb-640b-42d2-8c99-41f588351733\" (UID: \"d5f369eb-640b-42d2-8c99-41f588351733\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.926814    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\" (UniqueName: \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\") pod \"d2bb051a-7aa2-4367-9137-22d0db840c71\" (UID: \"d2bb051a-7aa2-4367-9137-22d0db840c71\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.926880    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\") pod \"4c47d360-0612-4329-81e2-f85b667ef6dd\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927042    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\" (UniqueName: \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\") pod \"4c47d360-0612-4329-81e2-f85b667ef6dd\" (UID: \"4c47d360-0612-4329-81e2-f85b667ef6dd\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927120    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\") pod \"73c59c5b-d898-4d57-a84c-f72f46719315\" (UID: \"73c59c5b-d898-4d57-a84c-f72f46719315\") "
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927143    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca" pod "d5f369eb-640b-42d2-8c99-41f588351733" (UID: "d5f369eb-640b-42d2-8c99-41f588351733"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927630    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca" pod "4c47d360-0612-4329-81e2-f85b667ef6dd" (UID: "4c47d360-0612-4329-81e2-f85b667ef6dd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927749    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d2bb051a-7aa2-4367-9137-22d0db840c71-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927771    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d5f369eb-640b-42d2-8c99-41f588351733-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927787    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c47d360-0612-4329-81e2-f85b667ef6dd-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.927937    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca" pod "73c59c5b-d898-4d57-a84c-f72f46719315" (UID: "73c59c5b-d898-4d57-a84c-f72f46719315"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.928669    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2" pod "d5f369eb-640b-42d2-8c99-41f588351733" (UID: "d5f369eb-640b-42d2-8c99-41f588351733"). InnerVolumeSpecName "kube-api-access-xxrs2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.929435    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf" pod "d2bb051a-7aa2-4367-9137-22d0db840c71" (UID: "d2bb051a-7aa2-4367-9137-22d0db840c71"). InnerVolumeSpecName "kube-api-access-tjvgf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:20.929737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.929568    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj" pod "73c59c5b-d898-4d57-a84c-f72f46719315" (UID: "73c59c5b-d898-4d57-a84c-f72f46719315"). InnerVolumeSpecName "kube-api-access-t2qfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:20.930717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:20.929820    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7" pod "4c47d360-0612-4329-81e2-f85b667ef6dd" (UID: "4c47d360-0612-4329-81e2-f85b667ef6dd"). InnerVolumeSpecName "kube-api-access-t8ps7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:21.029034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.028886    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8ps7\" (UniqueName: \"kubernetes.io/projected/4c47d360-0612-4329-81e2-f85b667ef6dd-kube-api-access-t8ps7\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:21.029034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.028924    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/73c59c5b-d898-4d57-a84c-f72f46719315-maas-api-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:21.029034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.028939    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxrs2\" (UniqueName: \"kubernetes.io/projected/d5f369eb-640b-42d2-8c99-41f588351733-kube-api-access-xxrs2\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:21.029034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.028955    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2qfj\" (UniqueName: \"kubernetes.io/projected/73c59c5b-d898-4d57-a84c-f72f46719315-kube-api-access-t2qfj\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:21.029034 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.028969    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjvgf\" (UniqueName: \"kubernetes.io/projected/d2bb051a-7aa2-4367-9137-22d0db840c71-kube-api-access-tjvgf\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:21.584706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.584674    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"
Nov 01 03:45:21.584914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.584672    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd" event={"ID":"d2bb051a-7aa2-4367-9137-22d0db840c71","Type":"ContainerDied","Data":"fd422f5ed7b1484d8b5550d4f4ff53d291fbde0f3a08a9c8e405c1c443b5a1ea"}
Nov 01 03:45:21.584914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.584807    4892 scope.go:122] "RemoveContainer" containerID="457e340534ff476020e75336692ac6463d7876836a07961729001fa9713f1a2b"
Nov 01 03:45:21.586309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.586289    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"
Nov 01 03:45:21.586389 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.586304    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77" event={"ID":"73c59c5b-d898-4d57-a84c-f72f46719315","Type":"ContainerDied","Data":"1f23190d36f835ecb2f140afa2011fd136b86266c6dbd6357e18fe0da15a00d3"}
Nov 01 03:45:21.587712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.587692    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"
Nov 01 03:45:21.587712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.587696    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr" event={"ID":"4c47d360-0612-4329-81e2-f85b667ef6dd","Type":"ContainerDied","Data":"e4279592892c40d9a2fb65cae7cd30f2060e93dc16926920b560d65dde6ce5af"}
Nov 01 03:45:21.589163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.589136    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg" event={"ID":"d5f369eb-640b-42d2-8c99-41f588351733","Type":"ContainerDied","Data":"c54c664b5157b724a7552574f1cc2abfc41d0370cad1c3139451904b525f58d9"}
Nov 01 03:45:21.589274 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.589209    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"
Nov 01 03:45:21.602505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.602483    4892 scope.go:122] "RemoveContainer" containerID="4fff42d6a062e980f3d2115e4d7bf7f25851f045cd4c142312a35826ea082393"
Nov 01 03:45:21.612534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.612508    4892 scope.go:122] "RemoveContainer" containerID="3b407e1f77cfb8e1762b941943ba1bb3f6e3c29a5511645d11caa289dd2b9136"
Nov 01 03:45:21.621294 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.621274    4892 scope.go:122] "RemoveContainer" containerID="aa759327a47f6f467a5f79bfd3e9ce3474582f423a0e4028b5108e56b823c192"
Nov 01 03:45:21.704936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.704901    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" path="/var/lib/kubelet/pods/0d762540-b099-47d2-ad61-4ffa8e589ac7/volumes"
Nov 01 03:45:21.705420 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.705392    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" path="/var/lib/kubelet/pods/389a0c21-02b9-4815-a797-60800f3fbc51/volumes"
Nov 01 03:45:21.751050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.751019    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:21.759447 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.759419    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-ded2d7-29891745-zjtcr"]
Nov 01 03:45:21.870695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.870608    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:21.875271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.875240    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-2906cb-29891745-flg77"]
Nov 01 03:45:21.984111 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.984075    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:21.990397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:21.990369    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-d84de6-29891745-plvdd"]
Nov 01 03:45:22.101463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.101431    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:22.107924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.107892    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w6-211380-29891745-g6tbg"]
Nov 01 03:45:22.199181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199101    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:45:22.199694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199680    4892 cpu_manager.go:414] "containerMap: removing container" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199694    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199705    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199710    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199720    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199725    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199735    4892 cpu_manager.go:414] "containerMap: removing container" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199739    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.199742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199745    4892 cpu_manager.go:414] "containerMap: removing container" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199750    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199755    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199760    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199767    4892 cpu_manager.go:414] "containerMap: removing container" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199772    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199781    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199786    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199795    4892 cpu_manager.go:414] "containerMap: removing container" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199799    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199805    4892 cpu_manager.go:414] "containerMap: removing container" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199808    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199818    4892 cpu_manager.go:414] "containerMap: removing container" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199822    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199830    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199835    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199841    4892 cpu_manager.go:414] "containerMap: removing container" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199846    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199855    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199859    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199867    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199872    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199880    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199884    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199968    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199989    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.199997    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200007    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200018    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200026    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200034    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200042    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200049    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200055    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200061    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200067    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:45:22.200049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200073    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:45:22.201100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200083    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" containerName="cleanup"
Nov 01 03:45:22.201100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200094    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:45:22.201100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200104    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:45:22.201100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.200113    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:45:22.204875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.204856    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.265720 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.265688    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a279c2-openshift-default-dockercfg-59wsl\""
Nov 01 03:45:22.266430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.266401    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:45:22.347927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.347879    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rptb\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.347936    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348085    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348138    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348285    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348371    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.348602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348429    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.351616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348620    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.351616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.348761    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450049    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450155 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450148    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450199    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rptb\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450235    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450294    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450335    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450373    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450412 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450412    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450440    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450456    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450755 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450699    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450913 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450853    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.450973 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.450908    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.451230 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.451210    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.452965 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.452941    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.453423 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.453400    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.512317 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.512280    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.512500 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.512359    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rptb\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\") pod \"e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") " pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.519025 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.518967    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:22.970881 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.970855    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:45:22.972558 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:45:22.972530    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4c279f_14b1_4873_99fa_0b443cc349f1.slice/crio-4a6bffb986d849cb4f25eed245d90e4fa23f54eb9c01b495eead4300634423a4 WatchSource:0}: Error finding container 4a6bffb986d849cb4f25eed245d90e4fa23f54eb9c01b495eead4300634423a4: Status 404 returned error can't find the container with id 4a6bffb986d849cb4f25eed245d90e4fa23f54eb9c01b495eead4300634423a4
Nov 01 03:45:22.975451 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.975418    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:45:22.975557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.975485    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:45:22.975557 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:22.975518    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:45:23.627329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.627288    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerStarted","Data":"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"}
Nov 01 03:45:23.627523 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.627350    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerStarted","Data":"4a6bffb986d849cb4f25eed245d90e4fa23f54eb9c01b495eead4300634423a4"}
Nov 01 03:45:23.702797 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.702764    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c47d360-0612-4329-81e2-f85b667ef6dd" path="/var/lib/kubelet/pods/4c47d360-0612-4329-81e2-f85b667ef6dd/volumes"
Nov 01 03:45:23.703254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.703240    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" path="/var/lib/kubelet/pods/73c59c5b-d898-4d57-a84c-f72f46719315/volumes"
Nov 01 03:45:23.703613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.703600    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" path="/var/lib/kubelet/pods/d2bb051a-7aa2-4367-9137-22d0db840c71/volumes"
Nov 01 03:45:23.704325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.704308    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5f369eb-640b-42d2-8c99-41f588351733" path="/var/lib/kubelet/pods/d5f369eb-640b-42d2-8c99-41f588351733/volumes"
Nov 01 03:45:23.799313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:23.799262    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podStartSLOduration=1.799246245 podStartE2EDuration="1.799246245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:45:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:45:23.797883644 +0000 UTC m=+2290.213783174" watchObservedRunningTime="2026-11-01 03:45:23.799246245 +0000 UTC m=+2290.215145774"
Nov 01 03:45:24.519747 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:24.519711    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:24.526036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:24.526011    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:24.635286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:24.635260    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:24.636466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:24.636440    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:45:27.612385 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:27.612344    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:45:43.315003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:43.312039    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:45:57.078767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:57.078732    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:45:57.615561 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:57.615533    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:45:58.379451 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.379275    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerName="main" containerID="cri-o://7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413" gracePeriod=45
Nov 01 03:45:58.679733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.679710    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:45:58.753484 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753450    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.753688 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753533    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.753688 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753653    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.753805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753709    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\" (UniqueName: \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.753805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753757    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\" (UniqueName: \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.753909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.753802    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\") pod \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\" (UID: \"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb\") "
Nov 01 03:45:58.754175 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.754146    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:45:58.754445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.754424    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:45:58.754445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.754435    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.754556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.754444    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:45:58.756452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.756426    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:45:58.756452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.756429    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "kube-api-access-rs64l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:45:58.756577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.756495    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs" pod "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" (UID: "90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:45:58.856055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.856016    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.856055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.856056    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.856255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.856072    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rs64l\" (UniqueName: \"kubernetes.io/projected/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-kube-api-access-rs64l\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.856255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.856087    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.856255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.856102    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:45:58.975067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.971250    4892 generic.go:356] "Generic (PLEG): container finished" podID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerID="7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413" exitCode=0
Nov 01 03:45:58.975067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.971299    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"
Nov 01 03:45:58.975067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.971330    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" event={"ID":"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb","Type":"ContainerDied","Data":"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"}
Nov 01 03:45:58.975067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.971376    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" event={"ID":"90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb","Type":"ContainerDied","Data":"e26edf6236df1712f6c8dbd89525f0976441fb7e5adfeefe8102b37eafac01dc"}
Nov 01 03:45:58.975067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.971398    4892 scope.go:122] "RemoveContainer" containerID="7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"
Nov 01 03:45:58.982117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.982097    4892 scope.go:122] "RemoveContainer" containerID="7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"
Nov 01 03:45:58.982409 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:45:58.982389    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413\": container with ID starting with 7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413 not found: ID does not exist" containerID="7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"
Nov 01 03:45:58.982466 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:58.982416    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413"} err="failed to get container status \"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413\": rpc error: code = NotFound desc = could not find container \"7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413\": container with ID starting with 7c68ba0e7be967d559cf3df0fe830ba756acc626d003a7cffdc6f93fee51a413 not found: ID does not exist"
Nov 01 03:45:59.152271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:59.152223    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:45:59.157199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:59.157176    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls"]
Nov 01 03:45:59.573512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:59.573469    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-fdbdfefd/e2e-lifecycle-model-fdbdfefd-kserve-5f9f587769-57sls" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerName="main" probeResult="failure" output="Get \"https://10.134.0.93:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 03:45:59.702620 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:45:59.702590    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" path="/var/lib/kubelet/pods/90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb/volumes"
Nov 01 03:46:04.069373 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:04.069337    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:46:04.180383 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:04.180350    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:46:04.741011 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:04.740944    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:04.741196 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:04.741046    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:05.014080 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.014045    4892 generic.go:356] "Generic (PLEG): container finished" podID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerID="b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e" exitCode=0
Nov 01 03:46:05.014291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.014123    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerDied","Data":"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"}
Nov 01 03:46:05.014679 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.014645    4892 scope.go:122] "RemoveContainer" containerID="b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"
Nov 01 03:46:05.016296 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.016272    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:46:05.016363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.016334    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:46:05.016400 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:05.016382    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:46:06.050002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:06.045266    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerStarted","Data":"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"}
Nov 01 03:46:06.108222 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:06.108190    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:46:06.108416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:06.108396    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:06.108495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:06.108448    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:07.107903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:07.107867    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:07.108472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:07.107934    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:08.108334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:08.108303    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:08.108734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:08.108357    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:09.069693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.069651    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing" containerID="cri-o://21f1dc8a5cd652df5a0986aa23fb1b38e98f6d3a8af2533ce5e41923ae7ac443" gracePeriod=25
Nov 01 03:46:09.107541 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.107501    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:09.107727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.107565    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:09.180996 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.180945    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing" containerID="cri-o://dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf" gracePeriod=25
Nov 01 03:46:09.499341 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.499249    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:46:09.499508 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:09.499256    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:46:10.108298 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:10.108262    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:10.108507 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:10.108320    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:11.107784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:11.107745    4892 patch_prober.go:29] interesting pod/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused" start-of-body=
Nov 01 03:46:11.108271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:11.107807    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.114:15021/healthz/ready\": dial tcp 10.134.0.114:15021: connect: connection refused"
Nov 01 03:46:12.099558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.099511    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main" containerID="cri-o://3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5" gracePeriod=45
Nov 01 03:46:12.113573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.113541    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:46:12.114055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.113914    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:46:12.114725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.114703    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:46:12.380850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.380823    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:46:12.440169 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440132    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.440357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440235    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\" (UniqueName: \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.440357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440273    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.440357 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440311    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.440529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440360    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.440529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.440425    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\" (UniqueName: \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\") pod \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\" (UID: \"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981\") "
Nov 01 03:46:12.443325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.443291    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:46:12.443734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.443690    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:46:12.443924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.443894    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:46:12.444251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.444215    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "kube-api-access-ds7dh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:46:12.451261 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.451190    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:46:12.452056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.452025    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm" pod "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" (UID: "e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:46:12.541936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.541900    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:12.541936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.541938    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:12.542195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.541953    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:12.542195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.541966    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:12.542195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.542010    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:12.542195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:12.542026    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ds7dh\" (UniqueName: \"kubernetes.io/projected/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981-kube-api-access-ds7dh\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:13.091107 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.091072    4892 generic.go:356] "Generic (PLEG): container finished" podID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerID="3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5" exitCode=0
Nov 01 03:46:13.091289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.091126    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"
Nov 01 03:46:13.091289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.091152    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" event={"ID":"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981","Type":"ContainerDied","Data":"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"}
Nov 01 03:46:13.091289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.091196    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z" event={"ID":"e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981","Type":"ContainerDied","Data":"b2cb1d4ea1aa3dec9f6fac7e646c8373b8b33671a4285c4c5f982ff14c128e79"}
Nov 01 03:46:13.091289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.091213    4892 scope.go:122] "RemoveContainer" containerID="3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"
Nov 01 03:46:13.102506 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.102308    4892 scope.go:122] "RemoveContainer" containerID="3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"
Nov 01 03:46:13.102598 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:46:13.102581    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5\": container with ID starting with 3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5 not found: ID does not exist" containerID="3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"
Nov 01 03:46:13.102645 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.102607    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5"} err="failed to get container status \"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5\": rpc error: code = NotFound desc = could not find container \"3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5\": container with ID starting with 3f9709dee3976dc44b8b4e439f3d6aa245c21516e665470149a74238a3daf4d5 not found: ID does not exist"
Nov 01 03:46:13.257114 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.257070    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:46:13.260119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.260093    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-c8cb9a/premium-w2-c8cb9a-kserve-d59774988-7n78z"]
Nov 01 03:46:13.702777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:13.702748    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" path="/var/lib/kubelet/pods/e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981/volumes"
Nov 01 03:46:19.498902 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:19.498862    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:46:19.499338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:19.498869    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:46:27.611667 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:46:27.611581    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:46:29.230788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.230752    4892 generic.go:356] "Generic (PLEG): container finished" podID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerID="21f1dc8a5cd652df5a0986aa23fb1b38e98f6d3a8af2533ce5e41923ae7ac443" exitCode=0
Nov 01 03:46:29.231185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.230820    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" event={"ID":"c33c7329-1dcd-4c77-8752-8cd96900ebb2","Type":"ContainerDied","Data":"21f1dc8a5cd652df5a0986aa23fb1b38e98f6d3a8af2533ce5e41923ae7ac443"}
Nov 01 03:46:29.347608 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.347538    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:46:29.437182 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.437153    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:46:29.468611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.468581    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\") pod \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") "
Nov 01 03:46:29.468815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.468756    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\" (UniqueName: \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\") pod \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\" (UID: \"c33c7329-1dcd-4c77-8752-8cd96900ebb2\") "
Nov 01 03:46:29.468926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.468830    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\" (UniqueName: \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\") pod \"708543c0-447c-493b-9026-926c02ce35e6\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") "
Nov 01 03:46:29.468926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.468911    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\") pod \"708543c0-447c-493b-9026-926c02ce35e6\" (UID: \"708543c0-447c-493b-9026-926c02ce35e6\") "
Nov 01 03:46:29.469077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.469051    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume" pod "c33c7329-1dcd-4c77-8752-8cd96900ebb2" (UID: "c33c7329-1dcd-4c77-8752-8cd96900ebb2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:46:29.469367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.469332    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume" pod "708543c0-447c-493b-9026-926c02ce35e6" (UID: "708543c0-447c-493b-9026-926c02ce35e6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:46:29.469617 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.469594    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/708543c0-447c-493b-9026-926c02ce35e6-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:29.469716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.469621    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c33c7329-1dcd-4c77-8752-8cd96900ebb2-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:29.471065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.471043    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c" pod "708543c0-447c-493b-9026-926c02ce35e6" (UID: "708543c0-447c-493b-9026-926c02ce35e6"). InnerVolumeSpecName "kube-api-access-xm25c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:46:29.471217 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.471199    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn" pod "c33c7329-1dcd-4c77-8752-8cd96900ebb2" (UID: "c33c7329-1dcd-4c77-8752-8cd96900ebb2"). InnerVolumeSpecName "kube-api-access-2knvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:46:29.570742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.570709    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2knvn\" (UniqueName: \"kubernetes.io/projected/c33c7329-1dcd-4c77-8752-8cd96900ebb2-kube-api-access-2knvn\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:29.570742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:29.570736    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm25c\" (UniqueName: \"kubernetes.io/projected/708543c0-447c-493b-9026-926c02ce35e6-kube-api-access-xm25c\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:46:30.237847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.237812    4892 generic.go:356] "Generic (PLEG): container finished" podID="708543c0-447c-493b-9026-926c02ce35e6" containerID="dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf" exitCode=0
Nov 01 03:46:30.238387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.237860    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"
Nov 01 03:46:30.238387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.237884    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" event={"ID":"708543c0-447c-493b-9026-926c02ce35e6","Type":"ContainerDied","Data":"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"}
Nov 01 03:46:30.238387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.237930    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn" event={"ID":"708543c0-447c-493b-9026-926c02ce35e6","Type":"ContainerDied","Data":"b5801a90436c00d8210912430ada16f2911727e8255b01887808e26e6d35eb5c"}
Nov 01 03:46:30.238387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.237948    4892 scope.go:122] "RemoveContainer" containerID="dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"
Nov 01 03:46:30.239334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.239316    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"
Nov 01 03:46:30.239409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.239359    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz" event={"ID":"c33c7329-1dcd-4c77-8752-8cd96900ebb2","Type":"ContainerDied","Data":"bc3c03d23564b24eed78c1b5bbba6c96f0f7ca76e11cfc125966b1eaa75be378"}
Nov 01 03:46:30.248177 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.248155    4892 scope.go:122] "RemoveContainer" containerID="dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"
Nov 01 03:46:30.248444 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:46:30.248425    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf\": container with ID starting with dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf not found: ID does not exist" containerID="dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"
Nov 01 03:46:30.248494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.248449    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf"} err="failed to get container status \"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf\": rpc error: code = NotFound desc = could not find container \"dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf\": container with ID starting with dee29ebbcf66d24ea04e3a6b21518368ba169dcb2d9ea9016646548fc33f8eaf not found: ID does not exist"
Nov 01 03:46:30.248494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.248464    4892 scope.go:122] "RemoveContainer" containerID="21f1dc8a5cd652df5a0986aa23fb1b38e98f6d3a8af2533ce5e41923ae7ac443"
Nov 01 03:46:30.404409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.404323    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:46:30.454574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.454542    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-c8cb9a-6d84697c7f-z22qn"]
Nov 01 03:46:30.565291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.565259    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:46:30.567707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:30.567679    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-c8cb9a-6fb7fd5959-8mkcz"]
Nov 01 03:46:31.701185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:31.701155    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="708543c0-447c-493b-9026-926c02ce35e6" path="/var/lib/kubelet/pods/708543c0-447c-493b-9026-926c02ce35e6/volumes"
Nov 01 03:46:31.701590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:31.701536    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" path="/var/lib/kubelet/pods/c33c7329-1dcd-4c77-8752-8cd96900ebb2/volumes"
Nov 01 03:46:42.708776 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.708736    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:46:42.709363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709346    4892 cpu_manager.go:414] "containerMap: removing container" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709364    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709386    4892 cpu_manager.go:414] "containerMap: removing container" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709391    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="389a0c21-02b9-4815-a797-60800f3fbc51" containerName="cleanup"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709402    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709407    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main"
Nov 01 03:46:42.709416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709416    4892 cpu_manager.go:414] "containerMap: removing container" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerName="main"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709421    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerName="main"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709431    4892 cpu_manager.go:414] "containerMap: removing container" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709435    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73c59c5b-d898-4d57-a84c-f72f46719315" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709444    4892 cpu_manager.go:414] "containerMap: removing container" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709449    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709458    4892 cpu_manager.go:414] "containerMap: removing container" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709463    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a3176562-080f-41c2-894c-6ed32fc86a07" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709476    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709480    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709583    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5f369eb-640b-42d2-8c99-41f588351733" containerName="cleanup"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709593    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="708543c0-447c-493b-9026-926c02ce35e6" containerName="payload-pre-processing"
Nov 01 03:46:42.709599 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709600    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d762540-b099-47d2-ad61-4ffa8e589ac7" containerName="cleanup"
Nov 01 03:46:42.710024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709608    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e46a4ca9-5b9e-4b28-b3c5-0b7a05db2981" containerName="main"
Nov 01 03:46:42.710024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709616    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="90889b05-c6c0-4f01-b3e2-5d95cc1ab4bb" containerName="main"
Nov 01 03:46:42.710024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709621    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2bb051a-7aa2-4367-9137-22d0db840c71" containerName="cleanup"
Nov 01 03:46:42.710024 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.709629    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="c33c7329-1dcd-4c77-8752-8cd96900ebb2" containerName="payload-processing"
Nov 01 03:46:42.711847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.711832    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.814488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.814444    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d84de6\"/\"openshift-service-ca.crt\""
Nov 01 03:46:42.814695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.814524    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d84de6\"/\"test-model-d84de6-kserve-self-signed-certs\""
Nov 01 03:46:42.814695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.814526    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-d84de6\"/\"kube-root-ca.crt\""
Nov 01 03:46:42.814695 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.814637    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-d84de6\"/\"default-dockercfg-mkzzb\""
Nov 01 03:46:42.817759 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.817735    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:46:42.837579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.837544    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8gdr\" (UniqueName: \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.837697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.837664    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.837869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.837844    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.838046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.838026    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.838144 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.838061    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.838144 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.838095    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939144    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939197    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939328    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8gdr\" (UniqueName: \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939445    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939544 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939511    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939600 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939559    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.939909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939882    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.940039 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.939956    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.940189 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.940166    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.941424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.941400    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:42.941905 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:42.941881    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:43.001254 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:43.001161    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8gdr\" (UniqueName: \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\") pod \"test-model-d84de6-kserve-566dcb7dd5-r2q5l\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") " pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:43.022718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:43.022685    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:43.408950 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:43.408928    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:46:43.411047 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:46:43.411016    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5177f8cc_cddb_426e_8afe_b8f70f045510.slice/crio-93d4a2ab350d80dadec04edb8baf9670fc4ce09c4acbafece87f18fe79bdcdae WatchSource:0}: Error finding container 93d4a2ab350d80dadec04edb8baf9670fc4ce09c4acbafece87f18fe79bdcdae: Status 404 returned error can't find the container with id 93d4a2ab350d80dadec04edb8baf9670fc4ce09c4acbafece87f18fe79bdcdae
Nov 01 03:46:44.350324 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:44.350282    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" event={"ID":"5177f8cc-cddb-426e-8afe-b8f70f045510","Type":"ContainerStarted","Data":"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"}
Nov 01 03:46:44.350746 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:44.350333    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" event={"ID":"5177f8cc-cddb-426e-8afe-b8f70f045510","Type":"ContainerStarted","Data":"93d4a2ab350d80dadec04edb8baf9670fc4ce09c4acbafece87f18fe79bdcdae"}
Nov 01 03:46:44.469445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:44.469395    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" podStartSLOduration=2.469380196 podStartE2EDuration="2.469380196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:46:44.46719525 +0000 UTC m=+2370.883094783" watchObservedRunningTime="2026-11-01 03:46:44.469380196 +0000 UTC m=+2370.885279725"
Nov 01 03:46:45.023144 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:45.023113    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:53.023318 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:53.023278    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:53.036163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:53.036136    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:53.420823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:46:53.420743    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:46:57.611804 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:46:57.611769    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:47:05.169091 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.169054    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:47:05.169497 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.169310    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy" containerID="cri-o://0423180833d8e09e0f835eca3072ae71782e2f91d07608aae4c56093ddc3938a" gracePeriod=30
Nov 01 03:47:05.218575 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.218538    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:47:05.218773 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.218752    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy" containerID="cri-o://ab887c4b9b131d3f56246dc222ba77cbf6bc1c37baf56357326d4cb0801eceeb" gracePeriod=30
Nov 01 03:47:05.308570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.308531    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:47:05.308803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:05.308724    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy" containerID="cri-o://8d97be2c36b9a294e96bd3f3e0b20bca232cae6404806e63953c7fac12d45746" gracePeriod=30
Nov 01 03:47:10.574055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.572861    4892 generic.go:356] "Generic (PLEG): container finished" podID="6cfb9067-0f82-439f-809d-35156a1542e3" containerID="0423180833d8e09e0f835eca3072ae71782e2f91d07608aae4c56093ddc3938a" exitCode=0
Nov 01 03:47:10.574055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.572926    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" event={"ID":"6cfb9067-0f82-439f-809d-35156a1542e3","Type":"ContainerDied","Data":"0423180833d8e09e0f835eca3072ae71782e2f91d07608aae4c56093ddc3938a"}
Nov 01 03:47:10.576579 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.576554    4892 generic.go:356] "Generic (PLEG): container finished" podID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerID="ab887c4b9b131d3f56246dc222ba77cbf6bc1c37baf56357326d4cb0801eceeb" exitCode=0
Nov 01 03:47:10.576693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.576627    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" event={"ID":"385d226c-e36e-4ac1-a508-f651fde5e2fe","Type":"ContainerDied","Data":"ab887c4b9b131d3f56246dc222ba77cbf6bc1c37baf56357326d4cb0801eceeb"}
Nov 01 03:47:10.578552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.578525    4892 generic.go:356] "Generic (PLEG): container finished" podID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerID="8d97be2c36b9a294e96bd3f3e0b20bca232cae6404806e63953c7fac12d45746" exitCode=0
Nov 01 03:47:10.578691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.578614    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" event={"ID":"1b4a3948-8dab-468a-8941-9bfd4644c2bb","Type":"ContainerDied","Data":"8d97be2c36b9a294e96bd3f3e0b20bca232cae6404806e63953c7fac12d45746"}
Nov 01 03:47:10.592379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.592126    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:47:10.683974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.683951    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:47:10.764444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764401    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.764655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764489    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.764655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764541    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.764655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764581    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.764655 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764640    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764676    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764706    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764740    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764782    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764813    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764850    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764882    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.764914    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\") pod \"385d226c-e36e-4ac1-a508-f651fde5e2fe\" (UID: \"385d226c-e36e-4ac1-a508-f651fde5e2fe\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.765050    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.765147    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.765177    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.765265 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.765247    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.766250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.765293    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\") pod \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\" (UID: \"1b4a3948-8dab-468a-8941-9bfd4644c2bb\") "
Nov 01 03:47:10.766363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.766316    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.766426 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.766359    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.767350 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.766490    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.767439 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767357    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.767513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767388    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:47:10.767614 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767589    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:47:10.768275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767694    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.768275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767873    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.768275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.767939    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:47:10.768275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.768160    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.768275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.768214    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.773518 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.773485    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:47:10.773805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.773775    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.776927 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.776895    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.777179 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.777149    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "kube-api-access-lkm8b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.777746 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.777714    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr" pod "1b4a3948-8dab-468a-8941-9bfd4644c2bb" (UID: "1b4a3948-8dab-468a-8941-9bfd4644c2bb"). InnerVolumeSpecName "kube-api-access-x9mtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.778264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.778227    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.805656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.784121    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy" pod "385d226c-e36e-4ac1-a508-f651fde5e2fe" (UID: "385d226c-e36e-4ac1-a508-f651fde5e2fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.867242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867210    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkm8b\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-kube-api-access-lkm8b\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867242 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867239    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/385d226c-e36e-4ac1-a508-f651fde5e2fe-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867249    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867258    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867267    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867276    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867285    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867293    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867302    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867310    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867319    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9mtr\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-kube-api-access-x9mtr\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867327    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867336    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867344    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867351    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867359    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/385d226c-e36e-4ac1-a508-f651fde5e2fe-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867368    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/385d226c-e36e-4ac1-a508-f651fde5e2fe-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.867434 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.867376    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b4a3948-8dab-468a-8941-9bfd4644c2bb-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.961176 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.961144    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:47:10.967744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967717    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.967879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967804    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.967879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967848    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967878    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967922    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.967964    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968029    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968173 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968060    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968192    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.968338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968203    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\") pod \"6cfb9067-0f82-439f-809d-35156a1542e3\" (UID: \"6cfb9067-0f82-439f-809d-35156a1542e3\") "
Nov 01 03:47:10.968445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968338    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968562    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968802    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968826    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968844    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.968962    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:47:10.969113 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.969084    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:10.970428 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.970403    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "kube-api-access-n2t4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.970654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.970624    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:10.970727 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.970652    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:47:10.971551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:10.971531    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy" pod "6cfb9067-0f82-439f-809d-35156a1542e3" (UID: "6cfb9067-0f82-439f-809d-35156a1542e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069751    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2t4x\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-kube-api-access-n2t4x\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069789    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6cfb9067-0f82-439f-809d-35156a1542e3-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069805    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6cfb9067-0f82-439f-809d-35156a1542e3-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069817    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069829    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6cfb9067-0f82-439f-809d-35156a1542e3-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.069849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.069843    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6cfb9067-0f82-439f-809d-35156a1542e3-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:11.587061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.587028    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"
Nov 01 03:47:11.587061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.587042    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2" event={"ID":"1b4a3948-8dab-468a-8941-9bfd4644c2bb","Type":"ContainerDied","Data":"0f235296bf4d8f4c39b3428d3e87a4e10e9553d7e5cdc9ed8ee1cca5e0b5adb9"}
Nov 01 03:47:11.587604 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.587115    4892 scope.go:122] "RemoveContainer" containerID="8d97be2c36b9a294e96bd3f3e0b20bca232cae6404806e63953c7fac12d45746"
Nov 01 03:47:11.588690 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.588590    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"
Nov 01 03:47:11.588690 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.588630    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z" event={"ID":"6cfb9067-0f82-439f-809d-35156a1542e3","Type":"ContainerDied","Data":"ccc35c8000bfec3aa0aacf43972c47d0ed839dfb60bd9b9762ed1e808d6b2f36"}
Nov 01 03:47:11.590356 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.590331    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn" event={"ID":"385d226c-e36e-4ac1-a508-f651fde5e2fe","Type":"ContainerDied","Data":"0ac6d8c445087c1dea93d6d56993081e9bec8d09312fe6f4308c86bce5d75441"}
Nov 01 03:47:11.590478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.590345    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"
Nov 01 03:47:11.598903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.598881    4892 scope.go:122] "RemoveContainer" containerID="0423180833d8e09e0f835eca3072ae71782e2f91d07608aae4c56093ddc3938a"
Nov 01 03:47:11.608733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.608714    4892 scope.go:122] "RemoveContainer" containerID="ab887c4b9b131d3f56246dc222ba77cbf6bc1c37baf56357326d4cb0801eceeb"
Nov 01 03:47:11.813797 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.813766    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:47:11.863316 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.863234    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-kbh6z"]
Nov 01 03:47:11.982662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.982629    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:47:11.989164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:11.989136    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-fbhrn"]
Nov 01 03:47:12.112565 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:12.112522    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:47:12.121051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:12.120948    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-mfwh2"]
Nov 01 03:47:13.702452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:13.702420    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" path="/var/lib/kubelet/pods/1b4a3948-8dab-468a-8941-9bfd4644c2bb/volumes"
Nov 01 03:47:13.702903 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:13.702888    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" path="/var/lib/kubelet/pods/385d226c-e36e-4ac1-a508-f651fde5e2fe/volumes"
Nov 01 03:47:13.703336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:13.703323    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" path="/var/lib/kubelet/pods/6cfb9067-0f82-439f-809d-35156a1542e3/volumes"
Nov 01 03:47:14.005209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:14.005185    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:47:14.011195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:14.011173    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:47:19.696459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.696368    4892 generic.go:356] "Generic (PLEG): container finished" podID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerID="8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4" exitCode=0
Nov 01 03:47:19.701849 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.701820    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerDied","Data":"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"}
Nov 01 03:47:19.702234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.702215    4892 scope.go:122] "RemoveContainer" containerID="8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"
Nov 01 03:47:19.703719 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.703694    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:19.703786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.703757    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:19.703822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:19.703790    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:20.704962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:20.704923    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerStarted","Data":"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"}
Nov 01 03:47:21.519647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:21.519614    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:47:21.535004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:21.521012    4892 patch_prober.go:29] interesting pod/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 01 03:47:21.535004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:21.521073    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 01 03:47:22.520263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:22.520224    4892 patch_prober.go:29] interesting pod/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 01 03:47:22.520752 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:22.520279    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 01 03:47:23.519715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:23.519683    4892 patch_prober.go:29] interesting pod/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 01 03:47:23.519915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:23.519732    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 01 03:47:24.519336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:24.519297    4892 patch_prober.go:29] interesting pod/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 01 03:47:24.519741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:24.519360    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 01 03:47:24.642124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:24.642088    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:47:25.519966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:25.519933    4892 patch_prober.go:29] interesting pod/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused" start-of-body=
Nov 01 03:47:25.520378 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:25.520001    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.124:15021/healthz/ready\": dial tcp 10.134.0.124:15021: connect: connection refused"
Nov 01 03:47:26.524450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:26.524418    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:47:26.525578 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:26.525555    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:47:27.611291 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:47:27.611253    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:47:28.785271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785240    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"]
Nov 01 03:47:28.785865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785840    4892 cpu_manager.go:414] "containerMap: removing container" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy"
Nov 01 03:47:28.785865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785860    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785874    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785879    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785906    4892 cpu_manager.go:414] "containerMap: removing container" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.785914    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.786045    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b4a3948-8dab-468a-8941-9bfd4644c2bb" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.786061    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="385d226c-e36e-4ac1-a508-f651fde5e2fe" containerName="istio-proxy"
Nov 01 03:47:28.786089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.786067    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cfb9067-0f82-439f-809d-35156a1542e3" containerName="istio-proxy"
Nov 01 03:47:28.788379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.788363    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.835549 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.835519    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"]
Nov 01 03:47:28.838970 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.838953    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"]
Nov 01 03:47:28.839083 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.839067    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:28.841865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.841846    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:28.883210 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.883183    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v7nmz\""
Nov 01 03:47:28.892189 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.892166    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"]
Nov 01 03:47:28.906765 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.906726    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.906916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.906848    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.906916 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.906873    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp8jt\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-kube-api-access-fp8jt\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.906966    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.907106    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.907146    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907280 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.907260    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907342 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.907304    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.907399 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.907366    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d0104a2-d41d-4052-9df0-9807d462c73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:28.954012 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.951276    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"]
Nov 01 03:47:28.955199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:28.955166    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"]
Nov 01 03:47:29.008661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008631    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.008828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008671    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtmqr\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-kube-api-access-wtmqr\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.008828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008705    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.008828 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008811    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.008937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008853    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.008937 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008893    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.009056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.008964    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d0104a2-d41d-4052-9df0-9807d462c73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009056 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009035    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009057    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.009147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009089    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lww9g\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-kube-api-access-lww9g\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.009147 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009132    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009165    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009223    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009257    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009277    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp8jt\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-kube-api-access-fp8jt\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009293    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.009311 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009312    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009346    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009392    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009570    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009622 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009609    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009627    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d0104a2-d41d-4052-9df0-9807d462c73e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009627    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.009826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009634    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.009826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009699    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.009826 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009790    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.010067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009877    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.010067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009909    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.010067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.009969    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.010067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.010034    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.010067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.010058    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.010234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.010163    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.011754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.011734    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.012253 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.012232    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.070534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.070458    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.070678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.070604    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp8jt\" (UniqueName: \"kubernetes.io/projected/1d0104a2-d41d-4052-9df0-9807d462c73e-kube-api-access-fp8jt\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-ghtkl\" (UID: \"1d0104a2-d41d-4052-9df0-9807d462c73e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.100891 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.100866    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:29.111379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111352    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111393    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111444    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111483 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111472    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111499    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111526    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111565    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111617    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111660    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111698    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111747    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111776    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtmqr\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-kube-api-access-wtmqr\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111787    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.111875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111822    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111882    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111929    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.111967    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lww9g\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-kube-api-access-lww9g\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112053    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112084    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112103    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.112195 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112160    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.112570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112424    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-workload-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.112570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112445    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.112718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112692    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.113013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.112963    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-credential-socket\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.113459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.113163    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-data\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.113576 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.113537    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-workload-certs\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.113846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.113821    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.115503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.115474    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.115610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.115570    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.116044 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.116020    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.117943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.117920    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-envoy\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.176901 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.176848    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.179106 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.179079    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lww9g\" (UniqueName: \"kubernetes.io/projected/9c23a0c8-ca80-4ef6-8d51-d2c7977c2762-kube-api-access-lww9g\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-gqkvd\" (UID: \"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.185062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.185018    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-istio-token\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.185390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.185366    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtmqr\" (UniqueName: \"kubernetes.io/projected/3b8b64b6-a777-4d47-9f48-4f52b5e9008e-kube-api-access-wtmqr\") pod \"maas-default-gateway-openshift-default-7c5b899d5f-wpzmc\" (UID: \"3b8b64b6-a777-4d47-9f48-4f52b5e9008e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.451772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.451671    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:29.458368 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.458346    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:29.537109 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.537079    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"]
Nov 01 03:47:29.545268 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:47:29.545227    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0104a2_d41d_4052_9df0_9807d462c73e.slice/crio-685050dfbe740baad0320bbca3221db925caf30e1d786cddbe89b1872c9cff51 WatchSource:0}: Error finding container 685050dfbe740baad0320bbca3221db925caf30e1d786cddbe89b1872c9cff51: Status 404 returned error can't find the container with id 685050dfbe740baad0320bbca3221db925caf30e1d786cddbe89b1872c9cff51
Nov 01 03:47:29.548062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.548027    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.548159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.548112    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.548234 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.548160    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.785560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.785522    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" event={"ID":"1d0104a2-d41d-4052-9df0-9807d462c73e","Type":"ContainerStarted","Data":"7e18aec0ee6f8402d498119e082f039292fba958485e36fb75b8690ad0996473"}
Nov 01 03:47:29.786070 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.785579    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" event={"ID":"1d0104a2-d41d-4052-9df0-9807d462c73e","Type":"ContainerStarted","Data":"685050dfbe740baad0320bbca3221db925caf30e1d786cddbe89b1872c9cff51"}
Nov 01 03:47:29.837051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.837022    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"]
Nov 01 03:47:29.837592 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:47:29.837566    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b8b64b6_a777_4d47_9f48_4f52b5e9008e.slice/crio-465231819ef1800fa4c69131b3888842eeebb26c083a5d52524b026dfc6b60a3 WatchSource:0}: Error finding container 465231819ef1800fa4c69131b3888842eeebb26c083a5d52524b026dfc6b60a3: Status 404 returned error can't find the container with id 465231819ef1800fa4c69131b3888842eeebb26c083a5d52524b026dfc6b60a3
Nov 01 03:47:29.840639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.840602    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.840734 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.840688    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.840771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.840735    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.886890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.886864    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"]
Nov 01 03:47:29.889205 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:47:29.889174    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c23a0c8_ca80_4ef6_8d51_d2c7977c2762.slice/crio-12fac6b5a583bdcf4b9ac5617a85737c576d0d9da0ecf1dc2e26b86d88c1b669 WatchSource:0}: Error finding container 12fac6b5a583bdcf4b9ac5617a85737c576d0d9da0ecf1dc2e26b86d88c1b669: Status 404 returned error can't find the container with id 12fac6b5a583bdcf4b9ac5617a85737c576d0d9da0ecf1dc2e26b86d88c1b669
Nov 01 03:47:29.891748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.891714    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.891842 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.891797    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.891928 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.891845    4892 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236060Ki","pods":"250"}
Nov 01 03:47:29.907791 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:29.907740    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podStartSLOduration=1.907721413 podStartE2EDuration="1.907721413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:47:29.906146194 +0000 UTC m=+2416.322045723" watchObservedRunningTime="2026-11-01 03:47:29.907721413 +0000 UTC m=+2416.323620943"
Nov 01 03:47:30.101672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.101584    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:30.103086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.103060    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:30.103214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.103107    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:30.807134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.807096    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" event={"ID":"3b8b64b6-a777-4d47-9f48-4f52b5e9008e","Type":"ContainerStarted","Data":"cf0379856048674ef75c1e2a210dbc44a497fb28f12d1cc62ac57c6894dd8e6b"}
Nov 01 03:47:30.807668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.807191    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" event={"ID":"3b8b64b6-a777-4d47-9f48-4f52b5e9008e","Type":"ContainerStarted","Data":"465231819ef1800fa4c69131b3888842eeebb26c083a5d52524b026dfc6b60a3"}
Nov 01 03:47:30.810100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.810047    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" event={"ID":"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762","Type":"ContainerStarted","Data":"6a4fb5eed3ad32f54bfb62aac963c963daf7eca0e53cbe70427e8b3f14fe9bb3"}
Nov 01 03:47:30.810100 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:30.810099    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" event={"ID":"9c23a0c8-ca80-4ef6-8d51-d2c7977c2762","Type":"ContainerStarted","Data":"12fac6b5a583bdcf4b9ac5617a85737c576d0d9da0ecf1dc2e26b86d88c1b669"}
Nov 01 03:47:31.086589 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.086462    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podStartSLOduration=3.086437646 podStartE2EDuration="3.086437646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:47:30.955276182 +0000 UTC m=+2417.371175694" watchObservedRunningTime="2026-11-01 03:47:31.086437646 +0000 UTC m=+2417.502337176"
Nov 01 03:47:31.087682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.087631    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podStartSLOduration=3.087615378 podStartE2EDuration="3.087615378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:47:31.075581644 +0000 UTC m=+2417.491481186" watchObservedRunningTime="2026-11-01 03:47:31.087615378 +0000 UTC m=+2417.503514910"
Nov 01 03:47:31.101514 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.101469    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:31.101687 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.101531    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:31.455942 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.455853    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:31.456338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.456316    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:31.456492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.456470    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:31.462649 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.462622    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:31.462944 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.462923    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:31.463124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:31.463099    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:32.101573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.101534    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:32.102095 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.101612    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:32.460062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.453148    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:32.460062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.453209    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:32.460062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.459335    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:32.460062 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:32.459390    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:33.105045 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.104531    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:33.105045 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.104606    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:33.454893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.454355    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:33.454893 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.454430    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:33.466603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.466570    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:33.466898 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:33.466835    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:34.102996 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.102427    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:34.102996 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.102499    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:34.454390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.454300    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:34.454964 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.454932    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:34.462748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.462718    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:34.462956 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:34.462933    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:35.101395 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.101357    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:35.101675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.101646    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:35.455471 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.455374    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:35.456077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.456044    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:35.460349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.460320    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:35.460537 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:35.460512    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:36.101271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.101239    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:36.101515 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.101494    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:36.453123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.453034    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:36.453123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.453112    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:36.459424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.459397    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:36.459751 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:36.459458    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:37.101340 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.101300    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused" start-of-body=
Nov 01 03:47:37.101525 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.101373    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": dial tcp 10.134.0.126:15021: connect: connection refused"
Nov 01 03:47:37.452838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.452743    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused" start-of-body=
Nov 01 03:47:37.452838 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.452810    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": dial tcp 10.134.0.127:15021: connect: connection refused"
Nov 01 03:47:37.458626 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.458602    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused" start-of-body=
Nov 01 03:47:37.458753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:37.458643    4892 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": dial tcp 10.134.0.128:15021: connect: connection refused"
Nov 01 03:47:39.085004 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.084265    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:39.085465 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.085238    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:39.086014 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.085960    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl"
Nov 01 03:47:39.173593 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.173565    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:39.173895 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.173873    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:39.174792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.174775    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc"
Nov 01 03:47:39.216453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.216425    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:39.217118 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.217009    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:39.217827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:39.217803    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd"
Nov 01 03:47:53.570793 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:53.570701    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:47:53.571404 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:53.570914    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" podUID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" containerName="maas-api" containerID="cri-o://1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4" gracePeriod=30
Nov 01 03:47:53.894306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:53.894273    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:47:54.030899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.030740    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\") pod \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") "
Nov 01 03:47:54.031148 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.030971    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") pod \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") "
Nov 01 03:47:54.031337 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.031183    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\") pod \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") "
Nov 01 03:47:54.031520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.031501    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\" (UniqueName: \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\") pod \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\" (UID: \"5f5b11c6-9f5f-4249-b723-3e0026576bf1\") "
Nov 01 03:47:54.041044 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.041015    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls" pod "5f5b11c6-9f5f-4249-b723-3e0026576bf1" (UID: "5f5b11c6-9f5f-4249-b723-3e0026576bf1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:47:54.041198 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.041180    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca" pod "5f5b11c6-9f5f-4249-b723-3e0026576bf1" (UID: "5f5b11c6-9f5f-4249-b723-3e0026576bf1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:47:54.049133 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.049101    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz" pod "5f5b11c6-9f5f-4249-b723-3e0026576bf1" (UID: "5f5b11c6-9f5f-4249-b723-3e0026576bf1"). InnerVolumeSpecName "kube-api-access-prlmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:47:54.055680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.055642    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs" pod "5f5b11c6-9f5f-4249-b723-3e0026576bf1" (UID: "5f5b11c6-9f5f-4249-b723-3e0026576bf1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:47:54.063221 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.063175    4892 generic.go:356] "Generic (PLEG): container finished" podID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" containerID="1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4" exitCode=0
Nov 01 03:47:54.063336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.063250    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"
Nov 01 03:47:54.063336 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.063288    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" event={"ID":"5f5b11c6-9f5f-4249-b723-3e0026576bf1","Type":"ContainerDied","Data":"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"}
Nov 01 03:47:54.063462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.063371    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4" event={"ID":"5f5b11c6-9f5f-4249-b723-3e0026576bf1","Type":"ContainerDied","Data":"87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f"}
Nov 01 03:47:54.063462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.063404    4892 scope.go:122] "RemoveContainer" containerID="1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"
Nov 01 03:47:54.083126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.083103    4892 scope.go:122] "RemoveContainer" containerID="1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"
Nov 01 03:47:54.083430 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:47:54.083411    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4\": container with ID starting with 1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4 not found: ID does not exist" containerID="1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"
Nov 01 03:47:54.083503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.083441    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4"} err="failed to get container status \"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4\": rpc error: code = NotFound desc = could not find container \"1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4\": container with ID starting with 1c5aa4856c11352cac244d3b76a3c6068a9c846aba047ba3f80490e1c275a8f4 not found: ID does not exist"
Nov 01 03:47:54.107096 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:47:54.107045    4892 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5b11c6_9f5f_4249_b723_3e0026576bf1.slice/crio-87406e78cb91cded52ed7f62085ca23f211f84c1781d67e0f0e68496d212868f\": RecentStats: unable to find data in memory cache]"
Nov 01 03:47:54.138306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.138259    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prlmz\" (UniqueName: \"kubernetes.io/projected/5f5b11c6-9f5f-4249-b723-3e0026576bf1-kube-api-access-prlmz\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:54.138306 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.138307    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f5b11c6-9f5f-4249-b723-3e0026576bf1-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:54.138574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.138326    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:54.138574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.138339    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f5b11c6-9f5f-4249-b723-3e0026576bf1-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:47:54.233482 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.233449    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:47:54.253920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:54.253889    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-c8cb9a-5f8cf9c8cf-js6j4"]
Nov 01 03:47:55.701825 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:47:55.701789    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" path="/var/lib/kubelet/pods/5f5b11c6-9f5f-4249-b723-3e0026576bf1/volumes"
Nov 01 03:47:57.610795 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:47:57.610762    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:48:02.839605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:02.839566    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:48:02.840172 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:02.839845    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" containerID="cri-o://22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797" gracePeriod=30
Nov 01 03:48:07.032510 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:07.032474    4892 patch_prober.go:29] interesting pod/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 03:48:07.032908 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:07.032545    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 03:48:08.104010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.103934    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:48:08.217675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.217630    4892 generic.go:356] "Generic (PLEG): container finished" podID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerID="22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797" exitCode=0
Nov 01 03:48:08.217883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.217706    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"
Nov 01 03:48:08.217883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.217713    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" event={"ID":"e2f2e9be-83a5-416f-96d5-4626208aad4e","Type":"ContainerDied","Data":"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"}
Nov 01 03:48:08.217883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.217817    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8" event={"ID":"e2f2e9be-83a5-416f-96d5-4626208aad4e","Type":"ContainerDied","Data":"76ace56c387de2ab68ef86c92220f4ba399ac29942e0b10b0374e6569c9e010a"}
Nov 01 03:48:08.217883 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.217850    4892 scope.go:122] "RemoveContainer" containerID="22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"
Nov 01 03:48:08.229216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.229190    4892 scope.go:122] "RemoveContainer" containerID="22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"
Nov 01 03:48:08.229542 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:08.229517    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797\": container with ID starting with 22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797 not found: ID does not exist" containerID="22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"
Nov 01 03:48:08.229634 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.229555    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797"} err="failed to get container status \"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797\": rpc error: code = NotFound desc = could not find container \"22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797\": container with ID starting with 22d2f28e5df400703e284aea419cc691965700cb3a9ad3f16bd5be33ec7b7797 not found: ID does not exist"
Nov 01 03:48:08.230869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.230844    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.230946 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.230901    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.230946    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231051 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231039    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231163    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231211    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231287    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:08.231310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231300    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231348    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231396    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\") pod \"e2f2e9be-83a5-416f-96d5-4626208aad4e\" (UID: \"e2f2e9be-83a5-416f-96d5-4626208aad4e\") "
Nov 01 03:48:08.231517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231408    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:48:08.231517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231420    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:08.231517 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231502    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:08.231658 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231619    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:08.231924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231906    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.231924 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231924    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2f2e9be-83a5-416f-96d5-4626208aad4e-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.232071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231935    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.232071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231949    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.232071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.231962    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.233151 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.233128    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:08.233444 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.233427    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:08.233548 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.233535    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "kube-api-access-mh8rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:08.234284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.234268    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo" pod "e2f2e9be-83a5-416f-96d5-4626208aad4e" (UID: "e2f2e9be-83a5-416f-96d5-4626208aad4e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:48:08.333396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.333363    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.333396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.333394    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.333396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.333405    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.333638 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.333414    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mh8rk\" (UniqueName: \"kubernetes.io/projected/e2f2e9be-83a5-416f-96d5-4626208aad4e-kube-api-access-mh8rk\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:08.639962 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.639926    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:48:08.648409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:08.648367    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-c8cb9a-openshift-default-5dbc574c7c-4swp8"]
Nov 01 03:48:09.701083 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:09.701049    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" path="/var/lib/kubelet/pods/e2f2e9be-83a5-416f-96d5-4626208aad4e/volumes"
Nov 01 03:48:19.089345 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:19.089315    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:48:19.195919 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:19.195678    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:48:24.089915 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.089873    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" podUID="4efd5f84-128f-4995-896a-ba381ada2b58" containerName="payload-processing" containerID="cri-o://c56a4ee6dd66af157e6d4ec1fbf6984c9855511e1601f63ce131103ba7f5df9b" gracePeriod=25
Nov 01 03:48:24.196813 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.196775    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" podUID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" containerName="payload-pre-processing" containerID="cri-o://26ddf1cd5d2cd74972eb2ef9c2e77735c09f7f56514c4af0db5c0ec7f36342c2" gracePeriod=25
Nov 01 03:48:24.321396 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.321359    4892 generic.go:356] "Generic (PLEG): container finished" podID="4efd5f84-128f-4995-896a-ba381ada2b58" containerID="c56a4ee6dd66af157e6d4ec1fbf6984c9855511e1601f63ce131103ba7f5df9b" exitCode=0
Nov 01 03:48:24.321537 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.321430    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" event={"ID":"4efd5f84-128f-4995-896a-ba381ada2b58","Type":"ContainerDied","Data":"c56a4ee6dd66af157e6d4ec1fbf6984c9855511e1601f63ce131103ba7f5df9b"}
Nov 01 03:48:24.323446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.323422    4892 generic.go:356] "Generic (PLEG): container finished" podID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" containerID="26ddf1cd5d2cd74972eb2ef9c2e77735c09f7f56514c4af0db5c0ec7f36342c2" exitCode=0
Nov 01 03:48:24.323559 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.323529    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" event={"ID":"6cf599c0-4906-496b-9b34-ad6c0175e3c0","Type":"ContainerDied","Data":"26ddf1cd5d2cd74972eb2ef9c2e77735c09f7f56514c4af0db5c0ec7f36342c2"}
Nov 01 03:48:24.370328 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.370302    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:48:24.457137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.457108    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\") pod \"4efd5f84-128f-4995-896a-ba381ada2b58\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") "
Nov 01 03:48:24.457335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.457143    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\" (UniqueName: \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\") pod \"4efd5f84-128f-4995-896a-ba381ada2b58\" (UID: \"4efd5f84-128f-4995-896a-ba381ada2b58\") "
Nov 01 03:48:24.457508 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.457479    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume" pod "4efd5f84-128f-4995-896a-ba381ada2b58" (UID: "4efd5f84-128f-4995-896a-ba381ada2b58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:48:24.457639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.457625    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4efd5f84-128f-4995-896a-ba381ada2b58-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:24.459281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.459258    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn" pod "4efd5f84-128f-4995-896a-ba381ada2b58" (UID: "4efd5f84-128f-4995-896a-ba381ada2b58"). InnerVolumeSpecName "kube-api-access-fcbjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:24.480560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.480536    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:48:24.558415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.558379    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\") pod \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") "
Nov 01 03:48:24.558651 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.558633    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\" (UniqueName: \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\") pod \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\" (UID: \"6cf599c0-4906-496b-9b34-ad6c0175e3c0\") "
Nov 01 03:48:24.558833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.558812    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume" pod "6cf599c0-4906-496b-9b34-ad6c0175e3c0" (UID: "6cf599c0-4906-496b-9b34-ad6c0175e3c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:48:24.559033 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.559016    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6cf599c0-4906-496b-9b34-ad6c0175e3c0-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:24.559081 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.559034    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcbjn\" (UniqueName: \"kubernetes.io/projected/4efd5f84-128f-4995-896a-ba381ada2b58-kube-api-access-fcbjn\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:24.560635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.560612    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj" pod "6cf599c0-4906-496b-9b34-ad6c0175e3c0" (UID: "6cf599c0-4906-496b-9b34-ad6c0175e3c0"). InnerVolumeSpecName "kube-api-access-8gshj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:24.660421 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:24.660328    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8gshj\" (UniqueName: \"kubernetes.io/projected/6cf599c0-4906-496b-9b34-ad6c0175e3c0-kube-api-access-8gshj\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:25.365827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.365724    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs" event={"ID":"4efd5f84-128f-4995-896a-ba381ada2b58","Type":"ContainerDied","Data":"a0a4403389ed75acf4b5977ac888a19b6b1b81ee1b5e7d3b227105c9c88833cf"}
Nov 01 03:48:25.365827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.365805    4892 scope.go:122] "RemoveContainer" containerID="c56a4ee6dd66af157e6d4ec1fbf6984c9855511e1601f63ce131103ba7f5df9b"
Nov 01 03:48:25.366438 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.366013    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"
Nov 01 03:48:25.379116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.378398    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4" event={"ID":"6cf599c0-4906-496b-9b34-ad6c0175e3c0","Type":"ContainerDied","Data":"da28506ef567729db0f5575cbc69db127f0cefacd86221bddb2d9b5ed25e43fd"}
Nov 01 03:48:25.379116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.378570    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"
Nov 01 03:48:25.402833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.401942    4892 scope.go:122] "RemoveContainer" containerID="26ddf1cd5d2cd74972eb2ef9c2e77735c09f7f56514c4af0db5c0ec7f36342c2"
Nov 01 03:48:25.536038 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.536003    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:48:25.585762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.585731    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-5fd6cf-665d5bb5d7-fm8b4"]
Nov 01 03:48:25.650015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.649921    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:48:25.655770 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.655745    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-5fd6cf-5cbdbffd6c-vxxvs"]
Nov 01 03:48:25.709796 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.709755    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4efd5f84-128f-4995-896a-ba381ada2b58" path="/var/lib/kubelet/pods/4efd5f84-128f-4995-896a-ba381ada2b58/volumes"
Nov 01 03:48:25.710477 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:25.710447    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" path="/var/lib/kubelet/pods/6cf599c0-4906-496b-9b34-ad6c0175e3c0/volumes"
Nov 01 03:48:27.610960 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:27.610923    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:48:36.962966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:36.962925    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:48:37.012914 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:37.012880    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:48:37.015446 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:37.015412    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:48:45.558652 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:45.558616    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:48:45.681639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:45.681013    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:48:50.559967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.559916    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" podUID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" containerName="payload-processing" containerID="cri-o://d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95" gracePeriod=25
Nov 01 03:48:50.681940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.681905    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" podUID="a4a54616-901a-4755-acf9-dbae83adc1ab" containerName="payload-pre-processing" containerID="cri-o://8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437" gracePeriod=25
Nov 01 03:48:50.824485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.824419    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:48:50.929745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.929712    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\" (UniqueName: \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\") pod \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") "
Nov 01 03:48:50.929936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.929831    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\") pod \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\" (UID: \"88f7fedd-f6eb-4cca-9f39-f30233f6fb90\") "
Nov 01 03:48:50.930303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.930234    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume" pod "88f7fedd-f6eb-4cca-9f39-f30233f6fb90" (UID: "88f7fedd-f6eb-4cca-9f39-f30233f6fb90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:48:50.930476 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.930457    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:50.931932 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.931905    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb" pod "88f7fedd-f6eb-4cca-9f39-f30233f6fb90" (UID: "88f7fedd-f6eb-4cca-9f39-f30233f6fb90"). InnerVolumeSpecName "kube-api-access-qq5rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:50.956441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:50.956413    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:48:51.031796 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.031762    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\" (UniqueName: \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\") pod \"a4a54616-901a-4755-acf9-dbae83adc1ab\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") "
Nov 01 03:48:51.032008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.031919    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\") pod \"a4a54616-901a-4755-acf9-dbae83adc1ab\" (UID: \"a4a54616-901a-4755-acf9-dbae83adc1ab\") "
Nov 01 03:48:51.032315 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.032285    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume" pod "a4a54616-901a-4755-acf9-dbae83adc1ab" (UID: "a4a54616-901a-4755-acf9-dbae83adc1ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:48:51.032552 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.032526    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a4a54616-901a-4755-acf9-dbae83adc1ab-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:51.032717 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.032694    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq5rb\" (UniqueName: \"kubernetes.io/projected/88f7fedd-f6eb-4cca-9f39-f30233f6fb90-kube-api-access-qq5rb\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:51.034010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.033974    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7" pod "a4a54616-901a-4755-acf9-dbae83adc1ab" (UID: "a4a54616-901a-4755-acf9-dbae83adc1ab"). InnerVolumeSpecName "kube-api-access-22rg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:51.134406 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.134326    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22rg7\" (UniqueName: \"kubernetes.io/projected/a4a54616-901a-4755-acf9-dbae83adc1ab-kube-api-access-22rg7\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:51.576538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.576508    4892 generic.go:356] "Generic (PLEG): container finished" podID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" containerID="d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95" exitCode=0
Nov 01 03:48:51.576931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.576589    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"
Nov 01 03:48:51.576931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.576586    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" event={"ID":"88f7fedd-f6eb-4cca-9f39-f30233f6fb90","Type":"ContainerDied","Data":"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"}
Nov 01 03:48:51.576931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.576801    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll" event={"ID":"88f7fedd-f6eb-4cca-9f39-f30233f6fb90","Type":"ContainerDied","Data":"c5c182f3394297d134f02d79853b81f4c193b22de8ba06c18f6b436af0607235"}
Nov 01 03:48:51.576931 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.576836    4892 scope.go:122] "RemoveContainer" containerID="d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"
Nov 01 03:48:51.578445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.578340    4892 generic.go:356] "Generic (PLEG): container finished" podID="a4a54616-901a-4755-acf9-dbae83adc1ab" containerID="8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437" exitCode=0
Nov 01 03:48:51.578531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.578452    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"
Nov 01 03:48:51.578531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.578469    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" event={"ID":"a4a54616-901a-4755-acf9-dbae83adc1ab","Type":"ContainerDied","Data":"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"}
Nov 01 03:48:51.578531 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.578505    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql" event={"ID":"a4a54616-901a-4755-acf9-dbae83adc1ab","Type":"ContainerDied","Data":"1f17a1c303511aea9751ebdde197554c38ab021b6e0ab217c30f2a6aed03cc29"}
Nov 01 03:48:51.588093 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.588075    4892 scope.go:122] "RemoveContainer" containerID="d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"
Nov 01 03:48:51.588415 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:51.588389    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95\": container with ID starting with d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95 not found: ID does not exist" containerID="d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"
Nov 01 03:48:51.588492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.588423    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95"} err="failed to get container status \"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95\": rpc error: code = NotFound desc = could not find container \"d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95\": container with ID starting with d748229f3c34368af570987ae7b5d57ff4b07dc4c0ecdf0d6eeb68d4c40bcb95 not found: ID does not exist"
Nov 01 03:48:51.588492 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.588446    4892 scope.go:122] "RemoveContainer" containerID="8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"
Nov 01 03:48:51.596850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.596830    4892 scope.go:122] "RemoveContainer" containerID="8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"
Nov 01 03:48:51.597119 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:51.597100    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437\": container with ID starting with 8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437 not found: ID does not exist" containerID="8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"
Nov 01 03:48:51.597181 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.597126    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437"} err="failed to get container status \"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437\": rpc error: code = NotFound desc = could not find container \"8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437\": container with ID starting with 8febee28e62f0e10995c04289f3568fe34430b64ac7d8afec3af32ec1eca3437 not found: ID does not exist"
Nov 01 03:48:51.751926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.751871    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:48:51.800943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.800910    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-2906cb-6669857b79-h4tql"]
Nov 01 03:48:51.875421 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.875335    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:48:51.878888 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.878862    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-2906cb-7bddbb5859-2wmll"]
Nov 01 03:48:51.983136 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:51.983091    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main" containerID="cri-o://e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6" gracePeriod=45
Nov 01 03:48:52.033178 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.033140    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" containerID="cri-o://17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9" gracePeriod=45
Nov 01 03:48:52.035235 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.035194    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main" containerID="cri-o://b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e" gracePeriod=45
Nov 01 03:48:52.129116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.129026    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main" probeResult="failure" output="Get \"https://10.134.0.48:8000/ready\": dial tcp 10.134.0.48:8000: connect: connection refused"
Nov 01 03:48:52.308442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.308028    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:48:52.328251 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.328224    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:48:52.337767 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.337742    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:48:52.348602 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348578    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.348748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348628    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.348748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348673    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.348874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348763    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.348874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348796    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.348874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348839    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.348874 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348866    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\" (UniqueName: \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348867    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348904    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.348918    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\" (UniqueName: \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349000    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349006    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349021    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349046    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\" (UniqueName: \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\") pod \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\" (UID: \"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd\") "
Nov 01 03:48:52.349102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349104    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.349505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349174    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\" (UniqueName: \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\") pod \"02dcb23a-9c75-4df2-befa-09f4b87607a5\" (UID: \"02dcb23a-9c75-4df2-befa-09f4b87607a5\") "
Nov 01 03:48:52.349505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349204    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.349753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349729    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.349840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349758    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.349840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349771    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.349840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349783    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.349840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349796    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.349840 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.349738    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.352472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352408    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:48:52.352472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352414    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "kube-api-access-zccdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:52.352472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352429    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.352472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352464    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs" pod "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" (UID: "5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:48:52.352691 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352560    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.352762 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.352743    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9" pod "02dcb23a-9c75-4df2-befa-09f4b87607a5" (UID: "02dcb23a-9c75-4df2-befa-09f4b87607a5"). InnerVolumeSpecName "kube-api-access-b8bl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:52.450461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450356    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.450461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450416    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.450682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450537    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.450682 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450597    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\" (UniqueName: \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.450800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450743    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.450800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.450787    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\" (UniqueName: \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\") pod \"70d0113f-df2b-48dc-91d7-821726cc5721\" (UID: \"70d0113f-df2b-48dc-91d7-821726cc5721\") "
Nov 01 03:48:52.451227 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451205    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451233    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zccdf\" (UniqueName: \"kubernetes.io/projected/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-kube-api-access-zccdf\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451249    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02dcb23a-9c75-4df2-befa-09f4b87607a5-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451261    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8bl9\" (UniqueName: \"kubernetes.io/projected/02dcb23a-9c75-4df2-befa-09f4b87607a5-kube-api-access-b8bl9\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451275    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451289    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02dcb23a-9c75-4df2-befa-09f4b87607a5-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451332 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451301    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.451635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451394    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.451635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451465    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.451635 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.451525    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.453124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.453095    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:48:52.453701 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.453672    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:48:52.453844 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.453807    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n" pod "70d0113f-df2b-48dc-91d7-821726cc5721" (UID: "70d0113f-df2b-48dc-91d7-821726cc5721"). InnerVolumeSpecName "kube-api-access-m4d4n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:48:52.552188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552157    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.552188 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552185    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70d0113f-df2b-48dc-91d7-821726cc5721-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.552388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552193    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.552388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552210    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m4d4n\" (UniqueName: \"kubernetes.io/projected/70d0113f-df2b-48dc-91d7-821726cc5721-kube-api-access-m4d4n\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.552388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552219    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.552388 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.552227    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70d0113f-df2b-48dc-91d7-821726cc5721-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:48:52.590677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.590611    4892 generic.go:356] "Generic (PLEG): container finished" podID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerID="e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6" exitCode=0
Nov 01 03:48:52.591145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.590761    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" event={"ID":"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd","Type":"ContainerDied","Data":"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"}
Nov 01 03:48:52.591145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.590820    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44" event={"ID":"5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd","Type":"ContainerDied","Data":"aabe23cc22201dee9de6210d28e4eb19f2c19bb43d5bff38a041fbb5fd2a2964"}
Nov 01 03:48:52.591145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.590847    4892 scope.go:122] "RemoveContainer" containerID="e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"
Nov 01 03:48:52.591330 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.591311    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"
Nov 01 03:48:52.593649 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.593622    4892 generic.go:356] "Generic (PLEG): container finished" podID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerID="17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9" exitCode=0
Nov 01 03:48:52.593771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.593663    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"
Nov 01 03:48:52.593771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.593712    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" event={"ID":"02dcb23a-9c75-4df2-befa-09f4b87607a5","Type":"ContainerDied","Data":"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"}
Nov 01 03:48:52.593771 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.593743    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz" event={"ID":"02dcb23a-9c75-4df2-befa-09f4b87607a5","Type":"ContainerDied","Data":"93063057c6d646b0b25f4aed00bc68d37ea60c57a6e85d10f76ec8e0a2d3f816"}
Nov 01 03:48:52.598896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.598873    4892 generic.go:356] "Generic (PLEG): container finished" podID="70d0113f-df2b-48dc-91d7-821726cc5721" containerID="b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e" exitCode=0
Nov 01 03:48:52.599040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.598953    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" event={"ID":"70d0113f-df2b-48dc-91d7-821726cc5721","Type":"ContainerDied","Data":"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"}
Nov 01 03:48:52.599040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.599015    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw" event={"ID":"70d0113f-df2b-48dc-91d7-821726cc5721","Type":"ContainerDied","Data":"1babc2a0acf1921d8bd66deedf67933ef67283ec536f83d6a42ac78aef430585"}
Nov 01 03:48:52.599164 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.599079    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"
Nov 01 03:48:52.604650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.604536    4892 scope.go:122] "RemoveContainer" containerID="e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"
Nov 01 03:48:52.605006 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:52.604966    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6\": container with ID starting with e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6 not found: ID does not exist" containerID="e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"
Nov 01 03:48:52.605135 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.605112    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6"} err="failed to get container status \"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6\": rpc error: code = NotFound desc = could not find container \"e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6\": container with ID starting with e82a369d04221c3f18ff88b57b57f1e0605267d212053e3b30ed6322463403e6 not found: ID does not exist"
Nov 01 03:48:52.605216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.605137    4892 scope.go:122] "RemoveContainer" containerID="17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"
Nov 01 03:48:52.616089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.616068    4892 scope.go:122] "RemoveContainer" containerID="17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"
Nov 01 03:48:52.616343 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:52.616315    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9\": container with ID starting with 17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9 not found: ID does not exist" containerID="17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"
Nov 01 03:48:52.616430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.616352    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9"} err="failed to get container status \"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9\": rpc error: code = NotFound desc = could not find container \"17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9\": container with ID starting with 17bfd6799a8355c128860be005056998a9da805a61a4a8741e7eddbf86cea4b9 not found: ID does not exist"
Nov 01 03:48:52.616430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.616377    4892 scope.go:122] "RemoveContainer" containerID="b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"
Nov 01 03:48:52.625036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.625014    4892 scope.go:122] "RemoveContainer" containerID="b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"
Nov 01 03:48:52.625320 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:52.625297    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e\": container with ID starting with b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e not found: ID does not exist" containerID="b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"
Nov 01 03:48:52.625400 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.625329    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e"} err="failed to get container status \"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e\": rpc error: code = NotFound desc = could not find container \"b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e\": container with ID starting with b8e94708f75302b710f1b1a8a2c915ce3f41ed61cdeca102368ca55c3592422e not found: ID does not exist"
Nov 01 03:48:52.775209 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.775173    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:48:52.781985 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.781948    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/simulator-w3-d7f535-kserve-69ddfdfd94-67mrw"]
Nov 01 03:48:52.891381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.891347    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:48:52.895777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:52.895750    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/premium-w3-d7f535-kserve-6f5cd7f459-wv8vz"]
Nov 01 03:48:53.029376 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.029295    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:48:53.037582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.037556    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d7f535/distinct-2-w3-d7f535-kserve-c48846f65-dtg44"]
Nov 01 03:48:53.701787 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.701758    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" path="/var/lib/kubelet/pods/02dcb23a-9c75-4df2-befa-09f4b87607a5/volumes"
Nov 01 03:48:53.702250 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.702229    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" path="/var/lib/kubelet/pods/5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd/volumes"
Nov 01 03:48:53.702671 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.702656    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" path="/var/lib/kubelet/pods/70d0113f-df2b-48dc-91d7-821726cc5721/volumes"
Nov 01 03:48:53.703381 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.703361    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" path="/var/lib/kubelet/pods/88f7fedd-f6eb-4cca-9f39-f30233f6fb90/volumes"
Nov 01 03:48:53.703740 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:48:53.703728    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4a54616-901a-4755-acf9-dbae83adc1ab" path="/var/lib/kubelet/pods/a4a54616-901a-4755-acf9-dbae83adc1ab/volumes"
Nov 01 03:48:57.610340 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:48:57.610300    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:49:00.559375 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.559339    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:49:00.559798 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.559551    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" podUID="77324486-1a26-460b-8282-3cbfcfefa919" containerName="maas-api" containerID="cri-o://db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809" gracePeriod=30
Nov 01 03:49:00.857472 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.857443    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:49:00.964941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.964474    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\") pod \"77324486-1a26-460b-8282-3cbfcfefa919\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") "
Nov 01 03:49:00.964941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.964599    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\") pod \"77324486-1a26-460b-8282-3cbfcfefa919\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") "
Nov 01 03:49:00.964941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.964637    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") pod \"77324486-1a26-460b-8282-3cbfcfefa919\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") "
Nov 01 03:49:00.964941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.964688    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\" (UniqueName: \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\") pod \"77324486-1a26-460b-8282-3cbfcfefa919\" (UID: \"77324486-1a26-460b-8282-3cbfcfefa919\") "
Nov 01 03:49:00.973184 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.965754    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca" pod "77324486-1a26-460b-8282-3cbfcfefa919" (UID: "77324486-1a26-460b-8282-3cbfcfefa919"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:49:00.973911 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.973879    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs" pod "77324486-1a26-460b-8282-3cbfcfefa919" (UID: "77324486-1a26-460b-8282-3cbfcfefa919"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:49:00.995953 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.983770    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw" pod "77324486-1a26-460b-8282-3cbfcfefa919" (UID: "77324486-1a26-460b-8282-3cbfcfefa919"). InnerVolumeSpecName "kube-api-access-zdglw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:49:00.996584 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:00.996554    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls" pod "77324486-1a26-460b-8282-3cbfcfefa919" (UID: "77324486-1a26-460b-8282-3cbfcfefa919"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:49:01.066823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.066786    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:01.066823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.066819    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/77324486-1a26-460b-8282-3cbfcfefa919-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:01.066823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.066830    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/77324486-1a26-460b-8282-3cbfcfefa919-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:01.067090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.066839    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdglw\" (UniqueName: \"kubernetes.io/projected/77324486-1a26-460b-8282-3cbfcfefa919-kube-api-access-zdglw\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:01.659864 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.659832    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"
Nov 01 03:49:01.660363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.659863    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" event={"ID":"77324486-1a26-460b-8282-3cbfcfefa919","Type":"ContainerDied","Data":"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"}
Nov 01 03:49:01.660363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.659837    4892 generic.go:356] "Generic (PLEG): container finished" podID="77324486-1a26-460b-8282-3cbfcfefa919" containerID="db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809" exitCode=0
Nov 01 03:49:01.660363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.659924    4892 scope.go:122] "RemoveContainer" containerID="db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"
Nov 01 03:49:01.660363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.660006    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw" event={"ID":"77324486-1a26-460b-8282-3cbfcfefa919","Type":"ContainerDied","Data":"a7c5e854c2af0ca1b2ee72692d11ba5493e1af183627a5ed503e0967758a99f9"}
Nov 01 03:49:01.669812 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.669790    4892 scope.go:122] "RemoveContainer" containerID="db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"
Nov 01 03:49:01.670107 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:49:01.670086    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809\": container with ID starting with db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809 not found: ID does not exist" containerID="db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"
Nov 01 03:49:01.670182 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.670120    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809"} err="failed to get container status \"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809\": rpc error: code = NotFound desc = could not find container \"db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809\": container with ID starting with db54d2a15b5f73dfe0b95a03f8dce124fb66b12919c33b3fc5fbfd75c7f85809 not found: ID does not exist"
Nov 01 03:49:01.826363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.826328    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:49:01.876779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:01.876743    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-2c8373-cbb8d74f7-smqtw"]
Nov 01 03:49:03.701471 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:03.701436    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77324486-1a26-460b-8282-3cbfcfefa919" path="/var/lib/kubelet/pods/77324486-1a26-460b-8282-3cbfcfefa919/volumes"
Nov 01 03:49:23.502387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:23.502308    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:49:23.608432 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:23.608400    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:49:27.610033 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:49:27.610000    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:49:28.503469 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.503405    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" podUID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" containerName="payload-processing" containerID="cri-o://a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca" gracePeriod=25
Nov 01 03:49:28.609194 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.609155    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" podUID="6c4b90ab-e882-4124-b3df-65f815606e10" containerName="payload-pre-processing" containerID="cri-o://fcc33fc956008be14a1b18d115b73b24de29e8097c96f41cb550ffe44c5ff47d" gracePeriod=25
Nov 01 03:49:28.765670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.765599    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:49:28.828185 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.828153    4892 generic.go:356] "Generic (PLEG): container finished" podID="6c4b90ab-e882-4124-b3df-65f815606e10" containerID="fcc33fc956008be14a1b18d115b73b24de29e8097c96f41cb550ffe44c5ff47d" exitCode=0
Nov 01 03:49:28.828360 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.828225    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" event={"ID":"6c4b90ab-e882-4124-b3df-65f815606e10","Type":"ContainerDied","Data":"fcc33fc956008be14a1b18d115b73b24de29e8097c96f41cb550ffe44c5ff47d"}
Nov 01 03:49:28.829745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.829717    4892 generic.go:356] "Generic (PLEG): container finished" podID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" containerID="a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca" exitCode=0
Nov 01 03:49:28.829871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.829753    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" event={"ID":"140b948a-f5f0-4d16-b601-ea8f97b35cc8","Type":"ContainerDied","Data":"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"}
Nov 01 03:49:28.829871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.829761    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"
Nov 01 03:49:28.829871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.829807    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj" event={"ID":"140b948a-f5f0-4d16-b601-ea8f97b35cc8","Type":"ContainerDied","Data":"10097f8b6a61fc8b5e2cd8f9cac12186d1d74b2eacf282db98fe8e94b6d7d528"}
Nov 01 03:49:28.829871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.829838    4892 scope.go:122] "RemoveContainer" containerID="a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"
Nov 01 03:49:28.840669 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.840645    4892 scope.go:122] "RemoveContainer" containerID="a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"
Nov 01 03:49:28.840949 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:49:28.840929    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca\": container with ID starting with a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca not found: ID does not exist" containerID="a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"
Nov 01 03:49:28.841067 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.840957    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca"} err="failed to get container status \"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca\": rpc error: code = NotFound desc = could not find container \"a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca\": container with ID starting with a3297be4764336c8ea081f7a487f0f53feebbb7143c05240278c82d47b7fdbca not found: ID does not exist"
Nov 01 03:49:28.874545 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.874524    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:49:28.909869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.909837    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\") pod \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") "
Nov 01 03:49:28.910064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.909887    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\" (UniqueName: \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\") pod \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\" (UID: \"140b948a-f5f0-4d16-b601-ea8f97b35cc8\") "
Nov 01 03:49:28.910276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.910253    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume" pod "140b948a-f5f0-4d16-b601-ea8f97b35cc8" (UID: "140b948a-f5f0-4d16-b601-ea8f97b35cc8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:49:28.910425 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.910408    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/140b948a-f5f0-4d16-b601-ea8f97b35cc8-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:28.911966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:28.911943    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq" pod "140b948a-f5f0-4d16-b601-ea8f97b35cc8" (UID: "140b948a-f5f0-4d16-b601-ea8f97b35cc8"). InnerVolumeSpecName "kube-api-access-lszdq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:49:29.011504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.011463    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\") pod \"6c4b90ab-e882-4124-b3df-65f815606e10\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") "
Nov 01 03:49:29.011675 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.011554    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\" (UniqueName: \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\") pod \"6c4b90ab-e882-4124-b3df-65f815606e10\" (UID: \"6c4b90ab-e882-4124-b3df-65f815606e10\") "
Nov 01 03:49:29.011889 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.011865    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume" pod "6c4b90ab-e882-4124-b3df-65f815606e10" (UID: "6c4b90ab-e882-4124-b3df-65f815606e10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:49:29.011997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.011922    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lszdq\" (UniqueName: \"kubernetes.io/projected/140b948a-f5f0-4d16-b601-ea8f97b35cc8-kube-api-access-lszdq\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:29.013575 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.013551    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m" pod "6c4b90ab-e882-4124-b3df-65f815606e10" (UID: "6c4b90ab-e882-4124-b3df-65f815606e10"). InnerVolumeSpecName "kube-api-access-hwj5m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:49:29.112768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.112687    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6c4b90ab-e882-4124-b3df-65f815606e10-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:29.112768 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.112717    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwj5m\" (UniqueName: \"kubernetes.io/projected/6c4b90ab-e882-4124-b3df-65f815606e10-kube-api-access-hwj5m\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:49:29.251379 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.251343    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:49:29.256529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.256497    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-417aa8-84798965-r8tdj"]
Nov 01 03:49:29.701441 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.701408    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" path="/var/lib/kubelet/pods/140b948a-f5f0-4d16-b601-ea8f97b35cc8/volumes"
Nov 01 03:49:29.837707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.837629    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7" event={"ID":"6c4b90ab-e882-4124-b3df-65f815606e10","Type":"ContainerDied","Data":"3f82b66ccb52c0d922bb759041d0bf36bb1e83bda4c15d98baf0bb40dfdc3642"}
Nov 01 03:49:29.837707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.837663    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"
Nov 01 03:49:29.837707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:29.837686    4892 scope.go:122] "RemoveContainer" containerID="fcc33fc956008be14a1b18d115b73b24de29e8097c96f41cb550ffe44c5ff47d"
Nov 01 03:49:30.003049 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:30.003016    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:49:30.052670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:30.052629    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-417aa8-576c9687b7-b72p7"]
Nov 01 03:49:31.701077 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:49:31.701044    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c4b90ab-e882-4124-b3df-65f815606e10" path="/var/lib/kubelet/pods/6c4b90ab-e882-4124-b3df-65f815606e10/volumes"
Nov 01 03:49:57.612201 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:49:57.612160    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:50:12.522040 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:12.521997    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:50:12.522650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:12.522255    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy" containerID="cri-o://4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff" gracePeriod=30
Nov 01 03:50:17.789238 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.789176    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:50:17.880677 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880645    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.880846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880704    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.880846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880728    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.880846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880798    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.880846 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880825    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.881055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880865    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.881055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880894    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.881055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880928    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.881055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.880937    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:17.881268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881056    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:17.881268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881061    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\") pod \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\" (UID: \"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8\") "
Nov 01 03:50:17.881268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881187    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:17.881437 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881381    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:50:17.881539 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881511    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:17.881639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881622    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.881707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881642    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.881707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881652    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.881707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881665    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.881707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.881677    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.883338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.883316    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:50:17.883457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.883343    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "kube-api-access-66st8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:50:17.883623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.883602    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:17.883684 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.883618    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo" pod "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" (UID: "6c2924a3-74cb-46ca-9c69-3a9b8df0efc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:50:17.982847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.982813    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66st8\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-kube-api-access-66st8\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.982847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.982842    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.983072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.982860    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:17.983072 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:17.982869    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:18.161020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.160912    4892 generic.go:356] "Generic (PLEG): container finished" podID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerID="4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff" exitCode=0
Nov 01 03:50:18.161020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.160970    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"
Nov 01 03:50:18.161204 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.161023    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerDied","Data":"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"}
Nov 01 03:50:18.161204 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.161086    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph" event={"ID":"6c2924a3-74cb-46ca-9c69-3a9b8df0efc8","Type":"ContainerDied","Data":"7374d0fe69e165947fbec60400cd25d26b1eef53797873cc3426999525f6a12b"}
Nov 01 03:50:18.161204 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.161116    4892 scope.go:122] "RemoveContainer" containerID="4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"
Nov 01 03:50:18.171325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.171305    4892 scope.go:122] "RemoveContainer" containerID="33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"
Nov 01 03:50:18.179745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.179729    4892 scope.go:122] "RemoveContainer" containerID="4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"
Nov 01 03:50:18.180005 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:50:18.179971    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff\": container with ID starting with 4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff not found: ID does not exist" containerID="4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"
Nov 01 03:50:18.180059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.180017    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff"} err="failed to get container status \"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff\": rpc error: code = NotFound desc = could not find container \"4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff\": container with ID starting with 4ffe8265b7474d527269c9f2b43b67312bbb056036f6fc7559861169e19a93ff not found: ID does not exist"
Nov 01 03:50:18.180059 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.180036    4892 scope.go:122] "RemoveContainer" containerID="33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"
Nov 01 03:50:18.180248 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:50:18.180233    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3\": container with ID starting with 33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3 not found: ID does not exist" containerID="33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"
Nov 01 03:50:18.180286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.180254    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3"} err="failed to get container status \"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3\": rpc error: code = NotFound desc = could not find container \"33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3\": container with ID starting with 33a148a23ca465803f53752aeeeb226008e4b6e211cf0f52b70238a1a8735ff3 not found: ID does not exist"
Nov 01 03:50:18.327022 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.326975    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:50:18.331737 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:18.331707    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5fd6cf-openshift-default-85cfffbc85-5jvph"]
Nov 01 03:50:19.701089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:19.701055    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" path="/var/lib/kubelet/pods/6c2924a3-74cb-46ca-9c69-3a9b8df0efc8/volumes"
Nov 01 03:50:23.965318 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:23.965284    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:50:23.965754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:23.965490    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy" containerID="cri-o://b4a1397d3c8e839375547b179f947dcfd1bdaaf90e710483b5401d857b886383" gracePeriod=30
Nov 01 03:50:27.646622 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:50:27.646579    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:50:29.225933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.225903    4892 generic.go:356] "Generic (PLEG): container finished" podID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerID="b4a1397d3c8e839375547b179f947dcfd1bdaaf90e710483b5401d857b886383" exitCode=0
Nov 01 03:50:29.226314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.225960    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerDied","Data":"b4a1397d3c8e839375547b179f947dcfd1bdaaf90e710483b5401d857b886383"}
Nov 01 03:50:29.226314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.226027    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9" event={"ID":"5a4fc747-c727-4424-9ed9-7ab87724471d","Type":"ContainerDied","Data":"e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6"}
Nov 01 03:50:29.226314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.226045    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e276f1d1eaac949d677343162656e51ab920c1478da2a1fad34120fe80543bd6"
Nov 01 03:50:29.226314 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.226058    4892 scope.go:122] "RemoveContainer" containerID="7812abc0e90c4a291674d18e199706075e808f22cf180f0629480f538d516b27"
Nov 01 03:50:29.233431 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.233407    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:50:29.326630 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326600    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.326810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326679    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.326810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326706    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.326810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326736    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.326810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326785    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.326810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326802    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.327202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326826    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.327202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326896    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.327202 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.326956    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\") pod \"5a4fc747-c727-4424-9ed9-7ab87724471d\" (UID: \"5a4fc747-c727-4424-9ed9-7ab87724471d\") "
Nov 01 03:50:29.327377 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327206    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:29.327377 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327302    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:29.327493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327426    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:29.327493 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327368    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:50:29.327605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327510    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:29.327605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327522    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.327605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.327539    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.329326 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.329301    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:50:29.329457 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.329441    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "kube-api-access-kn58h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:50:29.329519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.329474    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:50:29.329560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.329515    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token" pod "5a4fc747-c727-4424-9ed9-7ab87724471d" (UID: "5a4fc747-c727-4424-9ed9-7ab87724471d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:50:29.428936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428901    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.428936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428931    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kn58h\" (UniqueName: \"kubernetes.io/projected/5a4fc747-c727-4424-9ed9-7ab87724471d-kube-api-access-kn58h\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.428936 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428940    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a4fc747-c727-4424-9ed9-7ab87724471d-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.429263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428949    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.429263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428957    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.429263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428966    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:29.429263 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:29.428973    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a4fc747-c727-4424-9ed9-7ab87724471d-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:50:30.233003 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:30.232952    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"
Nov 01 03:50:30.517917 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:30.517881    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:50:30.521598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:30.521565    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-2906cb-openshift-default-7d74684b8f-nr8d9"]
Nov 01 03:50:31.700139 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:50:31.700105    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" path="/var/lib/kubelet/pods/5a4fc747-c727-4424-9ed9-7ab87724471d/volumes"
Nov 01 03:50:57.612506 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:50:57.612423    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:51:14.158930 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:14.158892    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:51:14.159571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:14.159162    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy" containerID="cri-o://64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d" gracePeriod=30
Nov 01 03:51:14.494707 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:14.494626    4892 scope.go:122] "RemoveContainer" containerID="b4a1397d3c8e839375547b179f947dcfd1bdaaf90e710483b5401d857b886383"
Nov 01 03:51:19.431275 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.431215    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:51:19.511667 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511638    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.511886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511717    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.511886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511816    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.511886 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511862    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511923    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.511964    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512030    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512096    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:19.512142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512127    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512192    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:19.512403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512218    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:51:19.512403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512266    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\") pod \"eac89573-9530-4151-b20a-6e9e5ab79ef7\" (UID: \"eac89573-9530-4151-b20a-6e9e5ab79ef7\") "
Nov 01 03:51:19.512565 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512402    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:19.512565 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512538    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:19.512741 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512724    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.512805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512747    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.512805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512761    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eac89573-9530-4151-b20a-6e9e5ab79ef7-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.512805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512775    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.512805 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.512790    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.514604 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.514577    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:19.514716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.514623    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "kube-api-access-zfhbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:19.514716 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.514642    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:19.514831 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.514804    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo" pod "eac89573-9530-4151-b20a-6e9e5ab79ef7" (UID: "eac89573-9530-4151-b20a-6e9e5ab79ef7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:51:19.515307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.515283    4892 generic.go:356] "Generic (PLEG): container finished" podID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerID="64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d" exitCode=0
Nov 01 03:51:19.515405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.515336    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"
Nov 01 03:51:19.515405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.515350    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerDied","Data":"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"}
Nov 01 03:51:19.515405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.515403    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz" event={"ID":"eac89573-9530-4151-b20a-6e9e5ab79ef7","Type":"ContainerDied","Data":"2c339d1f568ac9634ff3f3f75d9e24c5689b4f43db8afae3b228ee3cb8e05df5"}
Nov 01 03:51:19.515559 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.515425    4892 scope.go:122] "RemoveContainer" containerID="64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"
Nov 01 03:51:19.536498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.536474    4892 scope.go:122] "RemoveContainer" containerID="b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"
Nov 01 03:51:19.545909 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.545890    4892 scope.go:122] "RemoveContainer" containerID="64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"
Nov 01 03:51:19.546202 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:19.546180    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d\": container with ID starting with 64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d not found: ID does not exist" containerID="64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"
Nov 01 03:51:19.546273 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.546212    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d"} err="failed to get container status \"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d\": rpc error: code = NotFound desc = could not find container \"64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d\": container with ID starting with 64a58b45185e44751085570f389e10672d5dcb752a2275e6fdec6d16cd7bc48d not found: ID does not exist"
Nov 01 03:51:19.546273 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.546241    4892 scope.go:122] "RemoveContainer" containerID="b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"
Nov 01 03:51:19.546480 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:19.546464    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e\": container with ID starting with b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e not found: ID does not exist" containerID="b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"
Nov 01 03:51:19.546522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.546484    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e"} err="failed to get container status \"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e\": rpc error: code = NotFound desc = could not find container \"b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e\": container with ID starting with b89a3116b2425782050cd0a61f020ebdb7ef7f9f420748386ca65fdf42e5266e not found: ID does not exist"
Nov 01 03:51:19.553474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.553449    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:51:19.569405 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.569380    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:51:19.577276 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.577248    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:51:19.613706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.613677    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.613706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.613707    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfhbg\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-kube-api-access-zfhbg\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.613899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.613719    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.613899 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.613728    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eac89573-9530-4151-b20a-6e9e5ab79ef7-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:19.855705 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.855672    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:51:19.860313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:19.860281    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-417aa8-openshift-default-5745fc5847-z6hkz"]
Nov 01 03:51:21.700355 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:21.700327    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" path="/var/lib/kubelet/pods/eac89573-9530-4151-b20a-6e9e5ab79ef7/volumes"
Nov 01 03:51:22.258790 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.258755    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:51:22.259001 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.258961    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy" containerID="cri-o://b8deacb9e974e7885f0ceb6ae6cc2f7c2abfea4012462be7b2e14a170ba96362" gracePeriod=30
Nov 01 03:51:22.308262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.308230    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:51:22.308496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.308471    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy" containerID="cri-o://312952801d4d34707b8232d4cdea43eceee92d2182ad810bcd07a858fe8bafe2" gracePeriod=30
Nov 01 03:51:22.368008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.365436    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:51:22.368008 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:22.365745    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy" containerID="cri-o://874048c70c6cc89c7d0bb141cea3aa4d4347bc15edb68b67ab7d811d20b9460d" gracePeriod=30
Nov 01 03:51:27.570769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.570596    4892 generic.go:356] "Generic (PLEG): container finished" podID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerID="b8deacb9e974e7885f0ceb6ae6cc2f7c2abfea4012462be7b2e14a170ba96362" exitCode=0
Nov 01 03:51:27.571246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.570766    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerDied","Data":"b8deacb9e974e7885f0ceb6ae6cc2f7c2abfea4012462be7b2e14a170ba96362"}
Nov 01 03:51:27.571246 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.570863    4892 scope.go:122] "RemoveContainer" containerID="20b52f6116b2257226916bfaf33f72d6d75c456683470e5fbcaa31657ac82666"
Nov 01 03:51:27.574900 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.574873    4892 generic.go:356] "Generic (PLEG): container finished" podID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerID="874048c70c6cc89c7d0bb141cea3aa4d4347bc15edb68b67ab7d811d20b9460d" exitCode=0
Nov 01 03:51:27.575042 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.574968    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" event={"ID":"42581529-bc07-45a9-9bf1-1c45989b4bf6","Type":"ContainerDied","Data":"874048c70c6cc89c7d0bb141cea3aa4d4347bc15edb68b67ab7d811d20b9460d"}
Nov 01 03:51:27.577376 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.577350    4892 generic.go:356] "Generic (PLEG): container finished" podID="55edab28-38c0-4099-859e-1d59ed703620" containerID="312952801d4d34707b8232d4cdea43eceee92d2182ad810bcd07a858fe8bafe2" exitCode=0
Nov 01 03:51:27.577502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.577429    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" event={"ID":"55edab28-38c0-4099-859e-1d59ed703620","Type":"ContainerDied","Data":"312952801d4d34707b8232d4cdea43eceee92d2182ad810bcd07a858fe8bafe2"}
Nov 01 03:51:27.621142 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:27.621109    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:51:27.633808 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.633772    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:51:27.681120 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.681088    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:51:27.705089 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.705064    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:51:27.708335 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708313    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708358    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708382    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708402    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708461 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708446    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708484    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708578    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708617    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708661    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\") pod \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\" (UID: \"c53019ee-51d2-41a2-9b5f-45b0a23e992e\") "
Nov 01 03:51:27.708693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708674    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.708943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708799    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.708943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.708720    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:51:27.709083 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.709034    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.709142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.709082    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.709142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.709101    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.709142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.709126    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.709142 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.709140    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.710725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.710701    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:51:27.710836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.710814    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.711129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.711111    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.711129 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.711126    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h" pod "c53019ee-51d2-41a2-9b5f-45b0a23e992e" (UID: "c53019ee-51d2-41a2-9b5f-45b0a23e992e"). InnerVolumeSpecName "kube-api-access-6k52h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.809852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.809814    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.809877    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.809921    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.809961    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810009    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810036    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810102 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810099    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810130    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810174    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810180    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810204    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810230    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810275    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810296    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810314    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\") pod \"42581529-bc07-45a9-9bf1-1c45989b4bf6\" (UID: \"42581529-bc07-45a9-9bf1-1c45989b4bf6\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810354    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.810485 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810399    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810495    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810701    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810672    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810710    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810718    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810886    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810887    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810921    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.811090 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.810951    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\") pod \"55edab28-38c0-4099-859e-1d59ed703620\" (UID: \"55edab28-38c0-4099-859e-1d59ed703620\") "
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811495    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811516    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811531    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811544    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811544    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:51:27.811564 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811557    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/55edab28-38c0-4099-859e-1d59ed703620-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811582    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811600    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811613    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c53019ee-51d2-41a2-9b5f-45b0a23e992e-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811625    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811637    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811649    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811661    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k52h\" (UniqueName: \"kubernetes.io/projected/c53019ee-51d2-41a2-9b5f-45b0a23e992e-kube-api-access-6k52h\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811675    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c53019ee-51d2-41a2-9b5f-45b0a23e992e-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811688    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.812128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.811930    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.813643 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.813608    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.813904 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.813867    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.814068 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.814046    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:27.814268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.814115    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "kube-api-access-p5pdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.814268 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.814191    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "kube-api-access-48h5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.815512 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.815491    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:51:27.816694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.816674    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo" pod "42581529-bc07-45a9-9bf1-1c45989b4bf6" (UID: "42581529-bc07-45a9-9bf1-1c45989b4bf6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:51:27.816974 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.816945    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token" pod "55edab28-38c0-4099-859e-1d59ed703620" (UID: "55edab28-38c0-4099-859e-1d59ed703620"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913363    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48h5x\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-kube-api-access-48h5x\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913402    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913417    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5pdf\" (UniqueName: \"kubernetes.io/projected/55edab28-38c0-4099-859e-1d59ed703620-kube-api-access-p5pdf\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913431    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42581529-bc07-45a9-9bf1-1c45989b4bf6-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913443    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/55edab28-38c0-4099-859e-1d59ed703620-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913456    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913469    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/55edab28-38c0-4099-859e-1d59ed703620-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913483    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913496    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:27.913590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:27.913513    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42581529-bc07-45a9-9bf1-1c45989b4bf6-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:28.592583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.592485    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf" event={"ID":"c53019ee-51d2-41a2-9b5f-45b0a23e992e","Type":"ContainerDied","Data":"77500ec6a6764d8e8fc08ca68abc3c8947e3b8fe85bc6fe45cb8fc898461a2d8"}
Nov 01 03:51:28.592583 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.592569    4892 scope.go:122] "RemoveContainer" containerID="b8deacb9e974e7885f0ceb6ae6cc2f7c2abfea4012462be7b2e14a170ba96362"
Nov 01 03:51:28.593130 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.592683    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"
Nov 01 03:51:28.595264 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.595235    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"
Nov 01 03:51:28.595571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.595232    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw" event={"ID":"42581529-bc07-45a9-9bf1-1c45989b4bf6","Type":"ContainerDied","Data":"da47911bd5fca0473e77897f03ffbe1a633e28a5783495458620847dd60519c4"}
Nov 01 03:51:28.597054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.596962    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl" event={"ID":"55edab28-38c0-4099-859e-1d59ed703620","Type":"ContainerDied","Data":"d9b414d278518fe2542021a4f6326c8782530dd822db91a989d41a01ec1c52f3"}
Nov 01 03:51:28.597214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.597111    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"
Nov 01 03:51:28.615676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.608135    4892 scope.go:122] "RemoveContainer" containerID="874048c70c6cc89c7d0bb141cea3aa4d4347bc15edb68b67ab7d811d20b9460d"
Nov 01 03:51:28.626928 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.626908    4892 scope.go:122] "RemoveContainer" containerID="312952801d4d34707b8232d4cdea43eceee92d2182ad810bcd07a858fe8bafe2"
Nov 01 03:51:28.762519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.762448    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:51:28.809503 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.809474    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-19faa5-openshift-default-59c44967-9fxhl"]
Nov 01 03:51:28.880425 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.880323    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:51:28.885410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.885376    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-c22dd8-openshift-default-5879c8cc6f-llpcf"]
Nov 01 03:51:28.994401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.994369    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:51:28.998542 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:28.998519    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-ded2d7-openshift-default-5f59b5bd9-5shfw"]
Nov 01 03:51:29.700929 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:29.700893    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" path="/var/lib/kubelet/pods/42581529-bc07-45a9-9bf1-1c45989b4bf6/volumes"
Nov 01 03:51:29.701397 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:29.701381    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55edab28-38c0-4099-859e-1d59ed703620" path="/var/lib/kubelet/pods/55edab28-38c0-4099-859e-1d59ed703620/volumes"
Nov 01 03:51:29.701781 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:29.701768    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" path="/var/lib/kubelet/pods/c53019ee-51d2-41a2-9b5f-45b0a23e992e/volumes"
Nov 01 03:51:34.574685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.574638    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main" containerID="cri-o://22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887" gracePeriod=45
Nov 01 03:51:34.589725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.589623    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" containerID="cri-o://7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6" gracePeriod=45
Nov 01 03:51:34.596061 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.596031    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main" containerID="cri-o://3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e" gracePeriod=45
Nov 01 03:51:34.601711 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.601679    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main" probeResult="failure" output="Get \"https://10.134.0.54:8000/ready\": dial tcp 10.134.0.54:8000: connect: connection refused"
Nov 01 03:51:34.862016 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.861678    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:51:34.881329 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.881305    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:51:34.890488 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.890467    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:51:34.989163 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989134    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.989365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989180    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.989365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989228    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\" (UniqueName: \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.989365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989260    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\" (UniqueName: \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.989365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989311    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.989365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989353    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989381    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989422    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\" (UniqueName: \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989465    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\" (UniqueName: \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989460    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989495    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989582    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.989647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989613    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989658    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\") pod \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\" (UID: \"34b8a7c8-e317-46f2-9630-e42b59e8edd0\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989692    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989726    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989760    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\" (UniqueName: \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989787    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\") pod \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\" (UID: \"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348\") "
Nov 01 03:51:34.990030 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.989821    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\") pod \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\" (UID: \"ce314bdc-f27e-4a84-b968-7c77c2a51c03\") "
Nov 01 03:51:34.990334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.990266    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.990671    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.990742    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.990920    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.991082    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.991221    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.991322    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.991446    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.991896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.991629    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.992458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.992386    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:51:34.992723 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.992684    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.993057 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.993026    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:51:34.993281 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.993256    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "kube-api-access-mkmhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:34.994516 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.994493    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.994731 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.994703    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6" pod "34b8a7c8-e317-46f2-9630-e42b59e8edd0" (UID: "34b8a7c8-e317-46f2-9630-e42b59e8edd0"). InnerVolumeSpecName "kube-api-access-fgsw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:34.995255 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.995231    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "kube-api-access-bp5n6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:51:34.995668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.995643    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm" pod "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" (UID: "0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:51:34.996054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:34.996026    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs" pod "ce314bdc-f27e-4a84-b968-7c77c2a51c03" (UID: "ce314bdc-f27e-4a84-b968-7c77c2a51c03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091613    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091640    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091650    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091659    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgsw6\" (UniqueName: \"kubernetes.io/projected/34b8a7c8-e317-46f2-9630-e42b59e8edd0-kube-api-access-fgsw6\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091669    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091677    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091685    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091693    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.091693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091700    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091707    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091717    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091725    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkmhm\" (UniqueName: \"kubernetes.io/projected/ce314bdc-f27e-4a84-b968-7c77c2a51c03-kube-api-access-mkmhm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091733    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091741    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce314bdc-f27e-4a84-b968-7c77c2a51c03-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091749    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34b8a7c8-e317-46f2-9630-e42b59e8edd0-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091756    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.092154 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.091763    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp5n6\" (UniqueName: \"kubernetes.io/projected/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348-kube-api-access-bp5n6\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:51:35.639496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.639466    4892 generic.go:356] "Generic (PLEG): container finished" podID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerID="7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6" exitCode=0
Nov 01 03:51:35.639939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.639514    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"
Nov 01 03:51:35.639939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.639553    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" event={"ID":"ce314bdc-f27e-4a84-b968-7c77c2a51c03","Type":"ContainerDied","Data":"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"}
Nov 01 03:51:35.639939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.639600    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57" event={"ID":"ce314bdc-f27e-4a84-b968-7c77c2a51c03","Type":"ContainerDied","Data":"c023a8c01df0fd2203fad84008deb710df477f0b6121c42521e4735a8516c596"}
Nov 01 03:51:35.639939 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.639622    4892 scope.go:122] "RemoveContainer" containerID="7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"
Nov 01 03:51:35.640992 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.640927    4892 generic.go:356] "Generic (PLEG): container finished" podID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerID="3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e" exitCode=0
Nov 01 03:51:35.640992 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.640959    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" event={"ID":"34b8a7c8-e317-46f2-9630-e42b59e8edd0","Type":"ContainerDied","Data":"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"}
Nov 01 03:51:35.641126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.641010    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"
Nov 01 03:51:35.641126 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.641017    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm" event={"ID":"34b8a7c8-e317-46f2-9630-e42b59e8edd0","Type":"ContainerDied","Data":"01823214061ab632670cea3d82997ff10be06cf489444512365276801749a800"}
Nov 01 03:51:35.642490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.642459    4892 generic.go:356] "Generic (PLEG): container finished" podID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerID="22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887" exitCode=0
Nov 01 03:51:35.642623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.642503    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"
Nov 01 03:51:35.642623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.642537    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" event={"ID":"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348","Type":"ContainerDied","Data":"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"}
Nov 01 03:51:35.642623 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.642570    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb" event={"ID":"0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348","Type":"ContainerDied","Data":"5025057db627fc75dd7945dbe5fecc146b7b5c23b43e79c55d37c764b94fcb17"}
Nov 01 03:51:35.648803 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.648787    4892 scope.go:122] "RemoveContainer" containerID="7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"
Nov 01 03:51:35.649107 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:35.649082    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6\": container with ID starting with 7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6 not found: ID does not exist" containerID="7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"
Nov 01 03:51:35.649196 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.649114    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6"} err="failed to get container status \"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6\": rpc error: code = NotFound desc = could not find container \"7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6\": container with ID starting with 7bd13ca2f96effeb73f7f28b3a9e01b0bde4b7aa7070b09ad3d225977a8143b6 not found: ID does not exist"
Nov 01 03:51:35.649196 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.649130    4892 scope.go:122] "RemoveContainer" containerID="3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"
Nov 01 03:51:35.658505 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.658488    4892 scope.go:122] "RemoveContainer" containerID="3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"
Nov 01 03:51:35.658794 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:35.658774    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e\": container with ID starting with 3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e not found: ID does not exist" containerID="3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"
Nov 01 03:51:35.658860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.658800    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e"} err="failed to get container status \"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e\": rpc error: code = NotFound desc = could not find container \"3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e\": container with ID starting with 3fc3e3d688d6fede18112cb6e9005988a9116f8564955a281989136afa76196e not found: ID does not exist"
Nov 01 03:51:35.658860 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.658816    4892 scope.go:122] "RemoveContainer" containerID="22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"
Nov 01 03:51:35.666560 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.666534    4892 scope.go:122] "RemoveContainer" containerID="22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"
Nov 01 03:51:35.666792 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:35.666773    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887\": container with ID starting with 22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887 not found: ID does not exist" containerID="22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"
Nov 01 03:51:35.666858 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.666796    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887"} err="failed to get container status \"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887\": rpc error: code = NotFound desc = could not find container \"22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887\": container with ID starting with 22eff7edb2c3d65045ed88904de1a6500e7b306d68eec88ba74814ba0e9c7887 not found: ID does not exist"
Nov 01 03:51:35.803866 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.803832    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:51:35.853865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.853793    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/premium-w1-b12199-kserve-85d78c6fbc-qbmvm"]
Nov 01 03:51:35.973780 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.973750    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:51:35.983958 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:35.983930    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-w1-b12199-kserve-644bc7d5db-xhq57"]
Nov 01 03:51:36.115316 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:36.115233    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:51:36.142384 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:36.142355    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-b12199/distinct-2-w1-b12199-kserve-7579877bb9-v6xcb"]
Nov 01 03:51:37.700487 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:37.700452    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" path="/var/lib/kubelet/pods/0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348/volumes"
Nov 01 03:51:37.700871 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:37.700862    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" path="/var/lib/kubelet/pods/34b8a7c8-e317-46f2-9630-e42b59e8edd0/volumes"
Nov 01 03:51:37.701269 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:37.701256    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" path="/var/lib/kubelet/pods/ce314bdc-f27e-4a84-b968-7c77c2a51c03/volumes"
Nov 01 03:51:53.817892 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:51:53.817859    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:51:57.609320 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:51:57.609281    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:52:00.465529 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:00.465497    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:52:00.465961 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:00.465710    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy" containerID="cri-o://d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394" gracePeriod=30
Nov 01 03:52:05.715955 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.715920    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:52:05.803020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.802970    4892 generic.go:356] "Generic (PLEG): container finished" podID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerID="d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394" exitCode=0
Nov 01 03:52:05.803020 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.803010    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerDied","Data":"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"}
Nov 01 03:52:05.803277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.803039    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"
Nov 01 03:52:05.803277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.803072    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9" event={"ID":"1a4c279f-14b1-4873-99fa-0b443cc349f1","Type":"ContainerDied","Data":"4a6bffb986d849cb4f25eed245d90e4fa23f54eb9c01b495eead4300634423a4"}
Nov 01 03:52:05.803277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.803097    4892 scope.go:122] "RemoveContainer" containerID="d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"
Nov 01 03:52:05.812169 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.812137    4892 scope.go:122] "RemoveContainer" containerID="8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"
Nov 01 03:52:05.814572 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814554    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814581    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814599    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814698    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814742 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814736    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814780    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814833 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814820    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.814926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814810    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:05.814926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814858    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:05.814926 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814887    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.815108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.814948    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:05.815108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815018    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\") pod \"1a4c279f-14b1-4873-99fa-0b443cc349f1\" (UID: \"1a4c279f-14b1-4873-99fa-0b443cc349f1\") "
Nov 01 03:52:05.815271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815251    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:52:05.815271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815255    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:05.815486 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815467    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c279f-14b1-4873-99fa-0b443cc349f1-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.815556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815488    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.815556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815498    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.815556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815511    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.815556 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.815520    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.816966 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.816944    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:05.817134 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.817112    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:52:05.817208 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.817190    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "kube-api-access-2rptb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:05.818571 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.818544    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token" pod "1a4c279f-14b1-4873-99fa-0b443cc349f1" (UID: "1a4c279f-14b1-4873-99fa-0b443cc349f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:05.821519 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.821501    4892 scope.go:122] "RemoveContainer" containerID="d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"
Nov 01 03:52:05.821863 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:05.821844    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394\": container with ID starting with d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394 not found: ID does not exist" containerID="d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"
Nov 01 03:52:05.821935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.821871    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394"} err="failed to get container status \"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394\": rpc error: code = NotFound desc = could not find container \"d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394\": container with ID starting with d553da0f0ca70d2def57a134fb8f8ec8d656b6ef2cc3296ce518f1bd4015a394 not found: ID does not exist"
Nov 01 03:52:05.821935 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.821887    4892 scope.go:122] "RemoveContainer" containerID="8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"
Nov 01 03:52:05.822186 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:05.822167    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4\": container with ID starting with 8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4 not found: ID does not exist" containerID="8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"
Nov 01 03:52:05.822229 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.822193    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4"} err="failed to get container status \"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4\": rpc error: code = NotFound desc = could not find container \"8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4\": container with ID starting with 8b96e007a0174ea0d290f1447ff5d950eb821d8d4d581d2991b7489dfdd7faa4 not found: ID does not exist"
Nov 01 03:52:05.916338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.916306    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.916338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.916334    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rptb\" (UniqueName: \"kubernetes.io/projected/1a4c279f-14b1-4873-99fa-0b443cc349f1-kube-api-access-2rptb\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.916338 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.916344    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:05.916573 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:05.916353    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c279f-14b1-4873-99fa-0b443cc349f1-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:06.227117 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:06.227080    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:52:06.233258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:06.233230    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a279c2-openshift-default-54ff6c9fff-z9hc9"]
Nov 01 03:52:07.701270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:07.701237    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" path="/var/lib/kubelet/pods/1a4c279f-14b1-4873-99fa-0b443cc349f1/volumes"
Nov 01 03:52:08.837273 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:08.837207    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" podUID="5177f8cc-cddb-426e-8afe-b8f70f045510" containerName="main" containerID="cri-o://cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8" gracePeriod=45
Nov 01 03:52:09.054266 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054225    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"]
Nov 01 03:52:09.054685 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054671    4892 cpu_manager.go:414] "containerMap: removing container" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054687    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054709    4892 cpu_manager.go:414] "containerMap: removing container" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054714    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054723    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" containerName="payload-pre-processing"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054730    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" containerName="payload-pre-processing"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054741    4892 cpu_manager.go:414] "containerMap: removing container" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:09.054745 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054746    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054752    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054758    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054764    4892 cpu_manager.go:414] "containerMap: removing container" podUID="a4a54616-901a-4755-acf9-dbae83adc1ab" containerName="payload-pre-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054768    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4a54616-901a-4755-acf9-dbae83adc1ab" containerName="payload-pre-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054774    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054778    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054784    4892 cpu_manager.go:414] "containerMap: removing container" podUID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" containerName="payload-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054788    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" containerName="payload-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054796    4892 cpu_manager.go:414] "containerMap: removing container" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054800    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054811    4892 cpu_manager.go:414] "containerMap: removing container" podUID="77324486-1a26-460b-8282-3cbfcfefa919" containerName="maas-api"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054817    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77324486-1a26-460b-8282-3cbfcfefa919" containerName="maas-api"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054826    4892 cpu_manager.go:414] "containerMap: removing container" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054832    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054843    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" containerName="maas-api"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054847    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" containerName="maas-api"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054854    4892 cpu_manager.go:414] "containerMap: removing container" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054858    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054864    4892 cpu_manager.go:414] "containerMap: removing container" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054868    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054875    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054881    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054890    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6c4b90ab-e882-4124-b3df-65f815606e10" containerName="payload-pre-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054896    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c4b90ab-e882-4124-b3df-65f815606e10" containerName="payload-pre-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054904    4892 cpu_manager.go:414] "containerMap: removing container" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054908    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054914    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054918    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054923    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054928    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054938    4892 cpu_manager.go:414] "containerMap: removing container" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054942    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054949    4892 cpu_manager.go:414] "containerMap: removing container" podUID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" containerName="payload-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054954    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" containerName="payload-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054965    4892 cpu_manager.go:414] "containerMap: removing container" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.054971    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055002    4892 cpu_manager.go:414] "containerMap: removing container" podUID="4efd5f84-128f-4995-896a-ba381ada2b58" containerName="payload-processing"
Nov 01 03:52:09.054972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055007    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4efd5f84-128f-4995-896a-ba381ada2b58" containerName="payload-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055014    4892 cpu_manager.go:414] "containerMap: removing container" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055018    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055024    4892 cpu_manager.go:414] "containerMap: removing container" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055028    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055036    4892 cpu_manager.go:414] "containerMap: removing container" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055043    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055116    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f5b11c6-9f5f-4249-b723-3e0026576bf1" containerName="maas-api"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055125    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055132    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055137    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a22c5f8-69c6-4e75-ada2-f3fa2b321bdd" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055145    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c4b90ab-e882-4124-b3df-65f815606e10" containerName="payload-pre-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055149    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="02dcb23a-9c75-4df2-befa-09f4b87607a5" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055154    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055158    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a4fc747-c727-4424-9ed9-7ab87724471d" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055163    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055169    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055174    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="140b948a-f5f0-4d16-b601-ea8f97b35cc8" containerName="payload-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055179    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce314bdc-f27e-4a84-b968-7c77c2a51c03" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055184    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="4efd5f84-128f-4995-896a-ba381ada2b58" containerName="payload-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055191    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="34b8a7c8-e317-46f2-9630-e42b59e8edd0" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055199    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="55edab28-38c0-4099-859e-1d59ed703620" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055206    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="42581529-bc07-45a9-9bf1-1c45989b4bf6" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055211    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f5f6fc7-8cf1-4c2e-a6fc-d3dadd2c5348" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055218    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="70d0113f-df2b-48dc-91d7-821726cc5721" containerName="main"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055223    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6cf599c0-4906-496b-9b34-ad6c0175e3c0" containerName="payload-pre-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055228    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4a54616-901a-4755-acf9-dbae83adc1ab" containerName="payload-pre-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055233    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2f2e9be-83a5-416f-96d5-4626208aad4e" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055238    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="88f7fedd-f6eb-4cca-9f39-f30233f6fb90" containerName="payload-processing"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055244    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055250    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="77324486-1a26-460b-8282-3cbfcfefa919" containerName="maas-api"
Nov 01 03:52:09.056124 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.055256    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a4c279f-14b1-4873-99fa-0b443cc349f1" containerName="istio-proxy"
Nov 01 03:52:09.058199 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.058182    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.101416 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.101353    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:52:09.113442 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.113414    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:52:09.113670 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.113652    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:52:09.147528 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.147492    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.147697 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.147673    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmq8\" (UniqueName: \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.147772 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.147713    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.163843 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.163814    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"]
Nov 01 03:52:09.248921 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.248892    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.248994    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\" (UniqueName: \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249048    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249075    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249245 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249104    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249245 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249133    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\" (UniqueName: \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\") pod \"5177f8cc-cddb-426e-8afe-b8f70f045510\" (UID: \"5177f8cc-cddb-426e-8afe-b8f70f045510\") "
Nov 01 03:52:09.249245 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249200    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:09.249418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249330    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:09.249418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249351    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmq8\" (UniqueName: \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.249418 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249383    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.249637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249440    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.249637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249550    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.249637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249565    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.249801 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.249770    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:09.250367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.250341    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.251582 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.251543    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:52:09.251763 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.251737    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "kube-api-access-b8gdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:09.251847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.251749    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm" pod "5177f8cc-cddb-426e-8afe-b8f70f045510" (UID: "5177f8cc-cddb-426e-8afe-b8f70f045510"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:52:09.252197 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.252175    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.311997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.311944    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmq8\" (UniqueName: \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\") pod \"maas-api-revoke-sub-32b08a25-ljg4l\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.350786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.350755    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5177f8cc-cddb-426e-8afe-b8f70f045510-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.350786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.350780    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.350786 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.350789    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5177f8cc-cddb-426e-8afe-b8f70f045510-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.351050 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.350800    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8gdr\" (UniqueName: \"kubernetes.io/projected/5177f8cc-cddb-426e-8afe-b8f70f045510-kube-api-access-b8gdr\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:09.375725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.375624    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:09.752287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.752253    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"]
Nov 01 03:52:09.755869 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:52:09.755837    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc585305_e654_449e_be9f_a2178b78ed83.slice/crio-4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a WatchSource:0}: Error finding container 4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a: Status 404 returned error can't find the container with id 4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a
Nov 01 03:52:09.832239 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.832204    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l" event={"ID":"dc585305-e654-449e-be9f-a2178b78ed83","Type":"ContainerStarted","Data":"4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a"}
Nov 01 03:52:09.833603 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.833576    4892 generic.go:356] "Generic (PLEG): container finished" podID="5177f8cc-cddb-426e-8afe-b8f70f045510" containerID="cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8" exitCode=0
Nov 01 03:52:09.833733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.833620    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"
Nov 01 03:52:09.833733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.833658    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" event={"ID":"5177f8cc-cddb-426e-8afe-b8f70f045510","Type":"ContainerDied","Data":"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"}
Nov 01 03:52:09.833733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.833696    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l" event={"ID":"5177f8cc-cddb-426e-8afe-b8f70f045510","Type":"ContainerDied","Data":"93d4a2ab350d80dadec04edb8baf9670fc4ce09c4acbafece87f18fe79bdcdae"}
Nov 01 03:52:09.833733 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.833713    4892 scope.go:122] "RemoveContainer" containerID="cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"
Nov 01 03:52:09.842562 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.842379    4892 scope.go:122] "RemoveContainer" containerID="cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"
Nov 01 03:52:09.842821 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:09.842669    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8\": container with ID starting with cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8 not found: ID does not exist" containerID="cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"
Nov 01 03:52:09.842821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:09.842694    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8"} err="failed to get container status \"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8\": rpc error: code = NotFound desc = could not find container \"cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8\": container with ID starting with cce83b7fffee00191cdb7ee489f3123721e1526bf51c25d44612e480d52d21d8 not found: ID does not exist"
Nov 01 03:52:10.003662 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:10.003627    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:52:10.006411 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:10.006386    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-d84de6/test-model-d84de6-kserve-566dcb7dd5-r2q5l"]
Nov 01 03:52:10.840604 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:10.840529    4892 generic.go:356] "Generic (PLEG): container finished" podID="dc585305-e654-449e-be9f-a2178b78ed83" containerID="784dddcea7c66950ed6bae1dcc3b64e7d2f8bc63e54a1031146dee9f8191c285" exitCode=0
Nov 01 03:52:10.840779 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:10.840605    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l" event={"ID":"dc585305-e654-449e-be9f-a2178b78ed83","Type":"ContainerDied","Data":"784dddcea7c66950ed6bae1dcc3b64e7d2f8bc63e54a1031146dee9f8191c285"}
Nov 01 03:52:11.699789 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:11.699750    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5177f8cc-cddb-426e-8afe-b8f70f045510" path="/var/lib/kubelet/pods/5177f8cc-cddb-426e-8afe-b8f70f045510/volumes"
Nov 01 03:52:11.990108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:11.990083    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:12.076766 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.076731    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\") pod \"dc585305-e654-449e-be9f-a2178b78ed83\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") "
Nov 01 03:52:12.076920 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.076859    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\" (UniqueName: \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\") pod \"dc585305-e654-449e-be9f-a2178b78ed83\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") "
Nov 01 03:52:12.077006 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.076966    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\") pod \"dc585305-e654-449e-be9f-a2178b78ed83\" (UID: \"dc585305-e654-449e-be9f-a2178b78ed83\") "
Nov 01 03:52:12.077327 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.077304    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca" pod "dc585305-e654-449e-be9f-a2178b78ed83" (UID: "dc585305-e654-449e-be9f-a2178b78ed83"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:52:12.078882 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.078853    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca" pod "dc585305-e654-449e-be9f-a2178b78ed83" (UID: "dc585305-e654-449e-be9f-a2178b78ed83"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:52:12.079027 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.079007    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8" pod "dc585305-e654-449e-be9f-a2178b78ed83" (UID: "dc585305-e654-449e-be9f-a2178b78ed83"). InnerVolumeSpecName "kube-api-access-cbmq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:12.177847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.177801    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:12.177847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.177841    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/dc585305-e654-449e-be9f-a2178b78ed83-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:12.177847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.177855    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbmq8\" (UniqueName: \"kubernetes.io/projected/dc585305-e654-449e-be9f-a2178b78ed83-kube-api-access-cbmq8\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:12.852257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.852220    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l" event={"ID":"dc585305-e654-449e-be9f-a2178b78ed83","Type":"ContainerDied","Data":"4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a"}
Nov 01 03:52:12.852257 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.852245    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"
Nov 01 03:52:12.852712 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:12.852273    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4b8ec57a58d0c796576b5f37bbc12598a29b70046d7454a3b5d28d47d4b3168a"
Nov 01 03:52:14.040693 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:14.040655    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:52:14.044555 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:14.044534    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:52:19.252715 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:19.252674    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:52:24.253692 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.253578    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" podUID="d234b547-be01-4cc0-bab9-d0f57f9540f4" containerName="payload-processing" containerID="cri-o://b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b" gracePeriod=25
Nov 01 03:52:24.515991 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.515898    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:52:24.601460 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.601424    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\") pod \"d234b547-be01-4cc0-bab9-d0f57f9540f4\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") "
Nov 01 03:52:24.601628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.601490    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\" (UniqueName: \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\") pod \"d234b547-be01-4cc0-bab9-d0f57f9540f4\" (UID: \"d234b547-be01-4cc0-bab9-d0f57f9540f4\") "
Nov 01 03:52:24.601876 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.601851    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume" pod "d234b547-be01-4cc0-bab9-d0f57f9540f4" (UID: "d234b547-be01-4cc0-bab9-d0f57f9540f4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:52:24.603650 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.603627    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn" pod "d234b547-be01-4cc0-bab9-d0f57f9540f4" (UID: "d234b547-be01-4cc0-bab9-d0f57f9540f4"). InnerVolumeSpecName "kube-api-access-khrqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:24.702319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.702285    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d234b547-be01-4cc0-bab9-d0f57f9540f4-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:24.702319 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.702312    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khrqn\" (UniqueName: \"kubernetes.io/projected/d234b547-be01-4cc0-bab9-d0f57f9540f4-kube-api-access-khrqn\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:24.910834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.910757    4892 generic.go:356] "Generic (PLEG): container finished" podID="d234b547-be01-4cc0-bab9-d0f57f9540f4" containerID="b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b" exitCode=0
Nov 01 03:52:24.910834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.910800    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"
Nov 01 03:52:24.911064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.910800    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" event={"ID":"d234b547-be01-4cc0-bab9-d0f57f9540f4","Type":"ContainerDied","Data":"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"}
Nov 01 03:52:24.911064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.910930    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q" event={"ID":"d234b547-be01-4cc0-bab9-d0f57f9540f4","Type":"ContainerDied","Data":"066858d80e3e0ee4bdb68e739d921158de2781177575272f916edc21985d5059"}
Nov 01 03:52:24.911064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.910959    4892 scope.go:122] "RemoveContainer" containerID="b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"
Nov 01 03:52:24.920513 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.920493    4892 scope.go:122] "RemoveContainer" containerID="b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"
Nov 01 03:52:24.920774 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:24.920757    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b\": container with ID starting with b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b not found: ID does not exist" containerID="b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"
Nov 01 03:52:24.920823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:24.920784    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b"} err="failed to get container status \"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b\": rpc error: code = NotFound desc = could not find container \"b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b\": container with ID starting with b53be299d7bf75ae0653bbcdb4bc7d3c5b171648b7cb1f7eb87529068aabdc9b not found: ID does not exist"
Nov 01 03:52:25.076060 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:25.076027    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:52:25.125640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:25.125609    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w6-7cd000-7f6dd886f7-djp4q"]
Nov 01 03:52:25.699865 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:25.699834    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d234b547-be01-4cc0-bab9-d0f57f9540f4" path="/var/lib/kubelet/pods/d234b547-be01-4cc0-bab9-d0f57f9540f4/volumes"
Nov 01 03:52:27.610358 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:27.610321    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:52:45.515491 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:45.515452    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:52:45.628810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:45.628778    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:52:46.774872 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.774841    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"]
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775233    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5177f8cc-cddb-426e-8afe-b8f70f045510" containerName="main"
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775245    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5177f8cc-cddb-426e-8afe-b8f70f045510" containerName="main"
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775257    4892 cpu_manager.go:414] "containerMap: removing container" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775262    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c53019ee-51d2-41a2-9b5f-45b0a23e992e" containerName="istio-proxy"
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775278    4892 cpu_manager.go:414] "containerMap: removing container" podUID="dc585305-e654-449e-be9f-a2178b78ed83" containerName="revoke-keys"
Nov 01 03:52:46.775282 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775284    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc585305-e654-449e-be9f-a2178b78ed83" containerName="revoke-keys"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775290    4892 cpu_manager.go:414] "containerMap: removing container" podUID="d234b547-be01-4cc0-bab9-d0f57f9540f4" containerName="payload-processing"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775295    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d234b547-be01-4cc0-bab9-d0f57f9540f4" containerName="payload-processing"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775381    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="d234b547-be01-4cc0-bab9-d0f57f9540f4" containerName="payload-processing"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775394    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5177f8cc-cddb-426e-8afe-b8f70f045510" containerName="main"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775400    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="eac89573-9530-4151-b20a-6e9e5ab79ef7" containerName="istio-proxy"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775409    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="6c2924a3-74cb-46ca-9c69-3a9b8df0efc8" containerName="istio-proxy"
Nov 01 03:52:46.775478 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.775415    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc585305-e654-449e-be9f-a2178b78ed83" containerName="revoke-keys"
Nov 01 03:52:46.778522 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.778500    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:46.838972 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.838934    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"]
Nov 01 03:52:46.873271 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.873237    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:52:46.882896 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.882871    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:52:46.933443 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.933408    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:46.933443 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.933447    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:46.933706 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:46.933473    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tldfn\" (UniqueName: \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.035076 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.034963    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tldfn\" (UniqueName: \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.035237 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.035195    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.035283 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.035243    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.036054 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.036028    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.037730 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.037703    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.098481 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.098447    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tldfn\" (UniqueName: \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\") pod \"maas-api-revoke-sub-73db0618-55ccn\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.393119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.393029    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:47.778942 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:47.778916    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"]
Nov 01 03:52:47.781357 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:52:47.781323    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e18464f_0229_4f09_b792_84711159c93f.slice/crio-b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062 WatchSource:0}: Error finding container b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062: Status 404 returned error can't find the container with id b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062
Nov 01 03:52:48.032403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:48.032373    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn" event={"ID":"8e18464f-0229-4f09-b792-84711159c93f","Type":"ContainerStarted","Data":"b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062"}
Nov 01 03:52:49.045823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:49.043442    4892 generic.go:356] "Generic (PLEG): container finished" podID="8e18464f-0229-4f09-b792-84711159c93f" containerID="4f25c7e7b1794bfcf3b15c1416e5a895e87b24101bdcf6ef76f74c3152b1b152" exitCode=0
Nov 01 03:52:49.045823 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:49.043511    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn" event={"ID":"8e18464f-0229-4f09-b792-84711159c93f","Type":"ContainerDied","Data":"4f25c7e7b1794bfcf3b15c1416e5a895e87b24101bdcf6ef76f74c3152b1b152"}
Nov 01 03:52:50.182933 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.182898    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:50.269748 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.269711    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\") pod \"8e18464f-0229-4f09-b792-84711159c93f\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") "
Nov 01 03:52:50.269941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.269847    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\") pod \"8e18464f-0229-4f09-b792-84711159c93f\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") "
Nov 01 03:52:50.269941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.269905    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\" (UniqueName: \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\") pod \"8e18464f-0229-4f09-b792-84711159c93f\" (UID: \"8e18464f-0229-4f09-b792-84711159c93f\") "
Nov 01 03:52:50.270214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.270183    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca" pod "8e18464f-0229-4f09-b792-84711159c93f" (UID: "8e18464f-0229-4f09-b792-84711159c93f"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:52:50.271827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.271799    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca" pod "8e18464f-0229-4f09-b792-84711159c93f" (UID: "8e18464f-0229-4f09-b792-84711159c93f"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:52:50.271918 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.271867    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn" pod "8e18464f-0229-4f09-b792-84711159c93f" (UID: "8e18464f-0229-4f09-b792-84711159c93f"). InnerVolumeSpecName "kube-api-access-tldfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:52:50.370757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.370669    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/8e18464f-0229-4f09-b792-84711159c93f-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:50.370757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.370701    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tldfn\" (UniqueName: \"kubernetes.io/projected/8e18464f-0229-4f09-b792-84711159c93f-kube-api-access-tldfn\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:50.370757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.370715    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/8e18464f-0229-4f09-b792-84711159c93f-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:52:50.516474 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.516407    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing" containerID="cri-o://4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856" gracePeriod=25
Nov 01 03:52:50.629422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:50.629338    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing" containerID="cri-o://a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f" gracePeriod=25
Nov 01 03:52:51.053739 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:51.053706    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"
Nov 01 03:52:51.053923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:51.053743    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn" event={"ID":"8e18464f-0229-4f09-b792-84711159c93f","Type":"ContainerDied","Data":"b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062"}
Nov 01 03:52:51.053923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:51.053792    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6507d74f7a42e73b0f474401aeaabc0d6a9d10c8a46003c5222373663c1c062"
Nov 01 03:52:55.673223 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:55.673192    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:52:55.673753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:55.673408    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy" containerID="cri-o://2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed" gracePeriod=30
Nov 01 03:52:56.265267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:56.265216    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:52:56.265267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:52:56.265234    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:52:57.609679 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:52:57.609645    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:53:00.930654 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.930634    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:53:00.968462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968399    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968439    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968470    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968530    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968633 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968569    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968781 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968699    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968781 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968773    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968819    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.968890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968853    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:00.968890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968854    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:00.968890 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.968865    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\") pod \"e9fd8137-b090-4280-b60b-ed4416d09cef\" (UID: \"e9fd8137-b090-4280-b60b-ed4416d09cef\") "
Nov 01 03:53:00.969137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969070    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:00.969137 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969103    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:00.969452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969368    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9fd8137-b090-4280-b60b-ed4416d09cef-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:00.969452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969396    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:00.969452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969411    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:00.969452 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969442    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:00.969760 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.969477    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:00.970666 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.970645    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:53:00.970938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.970912    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:00.971135 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.971112    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:00.971289 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:00.971275    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h" pod "e9fd8137-b090-4280-b60b-ed4416d09cef" (UID: "e9fd8137-b090-4280-b60b-ed4416d09cef"). InnerVolumeSpecName "kube-api-access-lnd4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:01.070005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.069942    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:01.070005 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.069973    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9fd8137-b090-4280-b60b-ed4416d09cef-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:01.070214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.070016    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:01.070214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.070028    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:01.070214 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.070041    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnd4h\" (UniqueName: \"kubernetes.io/projected/e9fd8137-b090-4280-b60b-ed4416d09cef-kube-api-access-lnd4h\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:01.107836 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.107803    4892 generic.go:356] "Generic (PLEG): container finished" podID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerID="2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed" exitCode=0
Nov 01 03:53:01.108013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.107929    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" event={"ID":"e9fd8137-b090-4280-b60b-ed4416d09cef","Type":"ContainerDied","Data":"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"}
Nov 01 03:53:01.108013 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.107969    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9" event={"ID":"e9fd8137-b090-4280-b60b-ed4416d09cef","Type":"ContainerDied","Data":"67a1e913294f57578d102a8985671949337ad9558b9f7a1b234543934fd073c5"}
Nov 01 03:53:01.108123 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.108011    4892 scope.go:122] "RemoveContainer" containerID="2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"
Nov 01 03:53:01.108179 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.108129    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"
Nov 01 03:53:01.118422 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.118401    4892 scope.go:122] "RemoveContainer" containerID="2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"
Nov 01 03:53:01.118752 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:01.118731    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed\": container with ID starting with 2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed not found: ID does not exist" containerID="2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"
Nov 01 03:53:01.118811 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.118764    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed"} err="failed to get container status \"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed\": rpc error: code = NotFound desc = could not find container \"2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed\": container with ID starting with 2cc920fef9a7b03005c18c6be57768f71e7bef0f71fc9501aed0976f994fefed not found: ID does not exist"
Nov 01 03:53:01.273424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.273381    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:53:01.275818 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.275793    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-b12199-openshift-default-774cb8c567-tdrx9"]
Nov 01 03:53:01.699778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:01.699702    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" path="/var/lib/kubelet/pods/e9fd8137-b090-4280-b60b-ed4416d09cef/volumes"
Nov 01 03:53:02.819119 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:02.819084    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:53:02.819618 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:02.819388    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy" containerID="cri-o://6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c" gracePeriod=30
Nov 01 03:53:06.265728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:06.265679    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:53:06.265728 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:06.265706    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:53:08.083243 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.083218    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:53:08.135794 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135716    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.135943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135791    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.135943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135833    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.135943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135874    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.135943 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135914    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.136192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.135944    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.136192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136049    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.136192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136101    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.136192 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136149    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\") pod \"f9af3ab0-f673-44e4-abf0-a80e380650e3\" (UID: \"f9af3ab0-f673-44e4-abf0-a80e380650e3\") "
Nov 01 03:53:08.136410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136215    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:08.136410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136232    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:08.136410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136299    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:08.136410 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136304    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:08.136611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136481    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:08.136611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136499    4892 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f9af3ab0-f673-44e4-abf0-a80e380650e3-istiod-ca-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.136611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136523    4892 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.136611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136537    4892 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-data\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.136611 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.136551    4892 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-credential-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.138058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.138035    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:08.138256 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.138220    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 03:53:08.138502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.138480    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "kube-api-access-kp84z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:08.138502 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.138494    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token" pod "f9af3ab0-f673-44e4-abf0-a80e380650e3" (UID: "f9af3ab0-f673-44e4-abf0-a80e380650e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:08.143778 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.143751    4892 generic.go:356] "Generic (PLEG): container finished" podID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerID="6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c" exitCode=0
Nov 01 03:53:08.143879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.143794    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"
Nov 01 03:53:08.143879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.143824    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerDied","Data":"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"}
Nov 01 03:53:08.143879 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.143877    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn" event={"ID":"f9af3ab0-f673-44e4-abf0-a80e380650e3","Type":"ContainerDied","Data":"a1b8fd0da77dfab2f104e42eb65534bfbf371bae1881fdfbcb77125248053aae"}
Nov 01 03:53:08.144088 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.143898    4892 scope.go:122] "RemoveContainer" containerID="6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"
Nov 01 03:53:08.158180 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.158163    4892 scope.go:122] "RemoveContainer" containerID="288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"
Nov 01 03:53:08.166285 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.166263    4892 scope.go:122] "RemoveContainer" containerID="6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"
Nov 01 03:53:08.166565 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:08.166547    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c\": container with ID starting with 6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c not found: ID does not exist" containerID="6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"
Nov 01 03:53:08.166616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.166574    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c"} err="failed to get container status \"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c\": rpc error: code = NotFound desc = could not find container \"6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c\": container with ID starting with 6b9918688d24e044e61129d19beb9e125f66ee4ee7f341d2718da3d180078f3c not found: ID does not exist"
Nov 01 03:53:08.166616 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.166593    4892 scope.go:122] "RemoveContainer" containerID="288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"
Nov 01 03:53:08.166813 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:08.166793    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5\": container with ID starting with 288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5 not found: ID does not exist" containerID="288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"
Nov 01 03:53:08.166858 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.166820    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5"} err="failed to get container status \"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5\": rpc error: code = NotFound desc = could not find container \"288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5\": container with ID starting with 288a1bbafe023a562e4470c444ca94d4ec06cc51e59927edf1d58c7b0653d3a5 not found: ID does not exist"
Nov 01 03:53:08.237459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.237429    4892 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-podinfo\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.237459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.237452    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kp84z\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-kube-api-access-kp84z\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.237459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.237465    4892 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-workload-socket\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.237678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.237473    4892 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-token\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.237678 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.237482    4892 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f9af3ab0-f673-44e4-abf0-a80e380650e3-istio-envoy\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:08.306537 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.306506    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:53:08.368538 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.368508    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:53:08.375036 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:08.375010    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-d84de6-openshift-default-6d5cffcb55-ncgdn"]
Nov 01 03:53:09.700296 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:09.700265    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" path="/var/lib/kubelet/pods/f9af3ab0-f673-44e4-abf0-a80e380650e3/volumes"
Nov 01 03:53:10.277377 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.277341    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:53:10.277558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.277537    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" podUID="632be7bb-4a1f-43d9-8017-b858ddca490b" containerName="maas-api" containerID="cri-o://1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b" gracePeriod=30
Nov 01 03:53:10.593769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.593741    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:53:10.661307 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.661277    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") pod \"632be7bb-4a1f-43d9-8017-b858ddca490b\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") "
Nov 01 03:53:10.661463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.661367    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\") pod \"632be7bb-4a1f-43d9-8017-b858ddca490b\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") "
Nov 01 03:53:10.661463 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.661419    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\" (UniqueName: \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\") pod \"632be7bb-4a1f-43d9-8017-b858ddca490b\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") "
Nov 01 03:53:10.661605 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.661583    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\") pod \"632be7bb-4a1f-43d9-8017-b858ddca490b\" (UID: \"632be7bb-4a1f-43d9-8017-b858ddca490b\") "
Nov 01 03:53:10.662002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.661932    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca" pod "632be7bb-4a1f-43d9-8017-b858ddca490b" (UID: "632be7bb-4a1f-43d9-8017-b858ddca490b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:10.663534 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.663502    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls" pod "632be7bb-4a1f-43d9-8017-b858ddca490b" (UID: "632be7bb-4a1f-43d9-8017-b858ddca490b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:10.663852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.663836    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs" pod "632be7bb-4a1f-43d9-8017-b858ddca490b" (UID: "632be7bb-4a1f-43d9-8017-b858ddca490b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:10.663953 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.663863    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj" pod "632be7bb-4a1f-43d9-8017-b858ddca490b" (UID: "632be7bb-4a1f-43d9-8017-b858ddca490b"). InnerVolumeSpecName "kube-api-access-9vjvj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:10.762570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.762541    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.762570 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.762569    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vjvj\" (UniqueName: \"kubernetes.io/projected/632be7bb-4a1f-43d9-8017-b858ddca490b-kube-api-access-9vjvj\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.763058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.762579    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/632be7bb-4a1f-43d9-8017-b858ddca490b-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.763058 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.762590    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/632be7bb-4a1f-43d9-8017-b858ddca490b-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.764496 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.764474    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:53:10.863869 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.863835    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\" (UniqueName: \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\") pod \"dba86e65-6371-4c76-9e31-90badb06dd05\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") "
Nov 01 03:53:10.864064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.863905    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\") pod \"dba86e65-6371-4c76-9e31-90badb06dd05\" (UID: \"dba86e65-6371-4c76-9e31-90badb06dd05\") "
Nov 01 03:53:10.864401 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.864367    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume" pod "dba86e65-6371-4c76-9e31-90badb06dd05" (UID: "dba86e65-6371-4c76-9e31-90badb06dd05"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:10.866046 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.866025    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl" pod "dba86e65-6371-4c76-9e31-90badb06dd05" (UID: "dba86e65-6371-4c76-9e31-90badb06dd05"). InnerVolumeSpecName "kube-api-access-x6lcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:10.879968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.879946    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:53:10.964445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.964411    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\") pod \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") "
Nov 01 03:53:10.964637 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.964518    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\" (UniqueName: \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\") pod \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\" (UID: \"cd6f6bba-82b3-44b5-a686-7a6eb4341633\") "
Nov 01 03:53:10.964754 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.964738    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6lcl\" (UniqueName: \"kubernetes.io/projected/dba86e65-6371-4c76-9e31-90badb06dd05-kube-api-access-x6lcl\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.964821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.964745    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume" pod "cd6f6bba-82b3-44b5-a686-7a6eb4341633" (UID: "cd6f6bba-82b3-44b5-a686-7a6eb4341633"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:10.964821 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.964760    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dba86e65-6371-4c76-9e31-90badb06dd05-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:10.966520 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:10.966500    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq" pod "cd6f6bba-82b3-44b5-a686-7a6eb4341633" (UID: "cd6f6bba-82b3-44b5-a686-7a6eb4341633"). InnerVolumeSpecName "kube-api-access-wf6rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:11.065577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.065501    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd6f6bba-82b3-44b5-a686-7a6eb4341633-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:11.065577 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.065531    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf6rq\" (UniqueName: \"kubernetes.io/projected/cd6f6bba-82b3-44b5-a686-7a6eb4341633-kube-api-access-wf6rq\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:11.161116 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.161079    4892 generic.go:356] "Generic (PLEG): container finished" podID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerID="a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f" exitCode=0
Nov 01 03:53:11.161284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.161126    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"
Nov 01 03:53:11.161284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.161167    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" event={"ID":"cd6f6bba-82b3-44b5-a686-7a6eb4341633","Type":"ContainerDied","Data":"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"}
Nov 01 03:53:11.161284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.161226    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt" event={"ID":"cd6f6bba-82b3-44b5-a686-7a6eb4341633","Type":"ContainerDied","Data":"4d413363fa9507de60eedb846bf1210a251e19eb0ded61f42d933b04bfd10a6d"}
Nov 01 03:53:11.161284 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.161254    4892 scope.go:122] "RemoveContainer" containerID="a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"
Nov 01 03:53:11.162458 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.162435    4892 generic.go:356] "Generic (PLEG): container finished" podID="dba86e65-6371-4c76-9e31-90badb06dd05" containerID="4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856" exitCode=0
Nov 01 03:53:11.162551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.162480    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"
Nov 01 03:53:11.162551 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.162532    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" event={"ID":"dba86e65-6371-4c76-9e31-90badb06dd05","Type":"ContainerDied","Data":"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"}
Nov 01 03:53:11.162661 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.162577    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s" event={"ID":"dba86e65-6371-4c76-9e31-90badb06dd05","Type":"ContainerDied","Data":"ca314268e5cc5636313fcae89b45473c138fdd3f113d0dc776d7a47945b1a1be"}
Nov 01 03:53:11.163769 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.163750    4892 generic.go:356] "Generic (PLEG): container finished" podID="632be7bb-4a1f-43d9-8017-b858ddca490b" containerID="1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b" exitCode=0
Nov 01 03:53:11.163852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.163811    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" event={"ID":"632be7bb-4a1f-43d9-8017-b858ddca490b","Type":"ContainerDied","Data":"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"}
Nov 01 03:53:11.163852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.163838    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx" event={"ID":"632be7bb-4a1f-43d9-8017-b858ddca490b","Type":"ContainerDied","Data":"f1d62d564f3f1daead9610bad85262463a804438f57e5408d09db961ed4bfd16"}
Nov 01 03:53:11.163997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.163894    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"
Nov 01 03:53:11.172807 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.172767    4892 scope.go:122] "RemoveContainer" containerID="a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"
Nov 01 03:53:11.173091 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:11.173075    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f\": container with ID starting with a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f not found: ID does not exist" containerID="a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"
Nov 01 03:53:11.173152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.173101    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f"} err="failed to get container status \"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f\": rpc error: code = NotFound desc = could not find container \"a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f\": container with ID starting with a142613b688dd11f07f639d077645956b7e9c55abb2e2ff5b408009d8193433f not found: ID does not exist"
Nov 01 03:53:11.173152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.173116    4892 scope.go:122] "RemoveContainer" containerID="4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"
Nov 01 03:53:11.181390 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.181359    4892 scope.go:122] "RemoveContainer" containerID="4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"
Nov 01 03:53:11.181677 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:11.181657    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856\": container with ID starting with 4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856 not found: ID does not exist" containerID="4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"
Nov 01 03:53:11.181729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.181684    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856"} err="failed to get container status \"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856\": rpc error: code = NotFound desc = could not find container \"4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856\": container with ID starting with 4ddb09d36519f884a571ce23f7214e565eaf980dfb0595c6ddaf0fa4f8c8c856 not found: ID does not exist"
Nov 01 03:53:11.181729 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.181701    4892 scope.go:122] "RemoveContainer" containerID="1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"
Nov 01 03:53:11.189409 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.189394    4892 scope.go:122] "RemoveContainer" containerID="1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"
Nov 01 03:53:11.189648 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:11.189630    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b\": container with ID starting with 1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b not found: ID does not exist" containerID="1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"
Nov 01 03:53:11.189725 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.189659    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b"} err="failed to get container status \"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b\": rpc error: code = NotFound desc = could not find container \"1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b\": container with ID starting with 1116981d37c37a4c723c505a3aa81852ec56c03bb9faa5e1926776fb08487f3b not found: ID does not exist"
Nov 01 03:53:11.328017 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.327922    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:53:11.335569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.335544    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-ded2d7-78cfdd6fdf-7glgx"]
Nov 01 03:53:11.445498 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.445458    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:53:11.451193 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.450723    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-d84de6-65b899f7-jtx8s"]
Nov 01 03:53:11.562940 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.562899    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:53:11.567558 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.567522    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d84de6-659c486cdc-x7mbt"]
Nov 01 03:53:11.699504 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.699423    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="632be7bb-4a1f-43d9-8017-b858ddca490b" path="/var/lib/kubelet/pods/632be7bb-4a1f-43d9-8017-b858ddca490b/volumes"
Nov 01 03:53:11.699822 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.699807    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" path="/var/lib/kubelet/pods/cd6f6bba-82b3-44b5-a686-7a6eb4341633/volumes"
Nov 01 03:53:11.700168 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:11.700154    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" path="/var/lib/kubelet/pods/dba86e65-6371-4c76-9e31-90badb06dd05/volumes"
Nov 01 03:53:15.408923 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:15.408880    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:53:20.410099 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:20.410053    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing" containerID="cri-o://74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730" gracePeriod=25
Nov 01 03:53:23.154568 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154530    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"]
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154875    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154885    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154900    4892 cpu_manager.go:414] "containerMap: removing container" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154905    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154919    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154925    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154933    4892 cpu_manager.go:414] "containerMap: removing container" podUID="632be7bb-4a1f-43d9-8017-b858ddca490b" containerName="maas-api"
Nov 01 03:53:23.154938 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154938    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="632be7bb-4a1f-43d9-8017-b858ddca490b" containerName="maas-api"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154946    4892 cpu_manager.go:414] "containerMap: removing container" podUID="8e18464f-0229-4f09-b792-84711159c93f" containerName="revoke-keys"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154952    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8e18464f-0229-4f09-b792-84711159c93f" containerName="revoke-keys"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154965    4892 cpu_manager.go:414] "containerMap: removing container" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.154969    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155054    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd6f6bba-82b3-44b5-a686-7a6eb4341633" containerName="payload-pre-processing"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155064    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e9fd8137-b090-4280-b60b-ed4416d09cef" containerName="istio-proxy"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155070    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="632be7bb-4a1f-43d9-8017-b858ddca490b" containerName="maas-api"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155076    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="8e18464f-0229-4f09-b792-84711159c93f" containerName="revoke-keys"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155081    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155087    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="dba86e65-6371-4c76-9e31-90badb06dd05" containerName="payload-processing"
Nov 01 03:53:23.155233 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.155095    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:23.160010 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.159965    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.216656 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.216617    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"]
Nov 01 03:53:23.264065 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.264028    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:53:23.264228 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.264072    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:53:23.284203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.284170    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.284352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.284227    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqvwh\" (UniqueName: \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.284352 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.284299    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.325968 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.325933    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main" containerID="cri-o://af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5" gracePeriod=45
Nov 01 03:53:23.385703 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.385664    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.385857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.385729    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqvwh\" (UniqueName: \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.385857 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.385768    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.386453 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.386427    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.387967 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.387947    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.447385 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.447353    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqvwh\" (UniqueName: \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\") pod \"maas-api-revoke-sub-416a1cd5-xfqks\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.473309 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.473274    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:23.520636 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.520600    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:53:23.573258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.573233    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:53:23.688653 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688617    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\" (UniqueName: \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.688852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688673    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.688852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688710    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.688852 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688810    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.689047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688868    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.689047 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.688924    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\") pod \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\" (UID: \"5e6434e1-0f34-4169-ae1f-5efd29007ce0\") "
Nov 01 03:53:23.689296 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.689272    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:23.689562 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.689539    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:23.689639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.689607    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:23.692969 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.692609    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 03:53:23.693108 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.693010    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "kube-api-access-9q6wc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:23.695159 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.695131    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs" pod "5e6434e1-0f34-4169-ae1f-5efd29007ce0" (UID: "5e6434e1-0f34-4169-ae1f-5efd29007ce0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:23.790598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790564    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.790598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790593    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q6wc\" (UniqueName: \"kubernetes.io/projected/5e6434e1-0f34-4169-ae1f-5efd29007ce0-kube-api-access-9q6wc\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.790598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790603    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.790598 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790612    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.790875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790620    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.790875 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.790628    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e6434e1-0f34-4169-ae1f-5efd29007ce0-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:23.860591 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:23.860555    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"]
Nov 01 03:53:23.863160 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:53:23.863133    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74b8a90_9c22_408b_a6db_a5afa3ec2c9b.slice/crio-dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d WatchSource:0}: Error finding container dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d: Status 404 returned error can't find the container with id dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d
Nov 01 03:53:24.231694 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.231660    4892 generic.go:356] "Generic (PLEG): container finished" podID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerID="af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5" exitCode=0
Nov 01 03:53:24.232267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.231712    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"
Nov 01 03:53:24.232267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.231748    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" event={"ID":"5e6434e1-0f34-4169-ae1f-5efd29007ce0","Type":"ContainerDied","Data":"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"}
Nov 01 03:53:24.232267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.231799    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj" event={"ID":"5e6434e1-0f34-4169-ae1f-5efd29007ce0","Type":"ContainerDied","Data":"74c796d9f5f7f628bef3b98301568dca394f7f52b5a2cda330aa126bdfc9f28a"}
Nov 01 03:53:24.232267 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.231824    4892 scope.go:122] "RemoveContainer" containerID="af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"
Nov 01 03:53:24.233683 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.233657    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks" event={"ID":"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b","Type":"ContainerStarted","Data":"d5cd4b319c83cb4e913201a4812451f6c93eed5cdfbd929c5f1e7af0237d61a6"}
Nov 01 03:53:24.233800 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.233704    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks" event={"ID":"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b","Type":"ContainerStarted","Data":"dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d"}
Nov 01 03:53:24.240429 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.240412    4892 scope.go:122] "RemoveContainer" containerID="af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"
Nov 01 03:53:24.240685 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:24.240666    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5\": container with ID starting with af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5 not found: ID does not exist" containerID="af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"
Nov 01 03:53:24.240761 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.240694    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5"} err="failed to get container status \"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5\": rpc error: code = NotFound desc = could not find container \"af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5\": container with ID starting with af9699401b80b40eb8b27ca5f06f4916fd61307b4cc0c71eb6d1dd07750b87f5 not found: ID does not exist"
Nov 01 03:53:24.404964 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.404909    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks" podStartSLOduration=1.404895137 podStartE2EDuration="1.404895137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:53:24.403678061 +0000 UTC m=+2770.819577591" watchObservedRunningTime="2026-11-01 03:53:24.404895137 +0000 UTC m=+2770.820794666"
Nov 01 03:53:24.516832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.516801    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:53:24.521631 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:24.521608    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-c22dd8/premium-w5-c22dd8-kserve-7967d66f6d-zbmmj"]
Nov 01 03:53:25.240792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:25.240761    4892 generic.go:356] "Generic (PLEG): container finished" podID="b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" containerID="d5cd4b319c83cb4e913201a4812451f6c93eed5cdfbd929c5f1e7af0237d61a6" exitCode=0
Nov 01 03:53:25.241203 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:25.240817    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks" event={"ID":"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b","Type":"ContainerDied","Data":"d5cd4b319c83cb4e913201a4812451f6c93eed5cdfbd929c5f1e7af0237d61a6"}
Nov 01 03:53:25.699574 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:25.699543    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" path="/var/lib/kubelet/pods/5e6434e1-0f34-4169-ae1f-5efd29007ce0/volumes"
Nov 01 03:53:26.380757 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.380731    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:26.519183 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.519151    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\") pod \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") "
Nov 01 03:53:26.519363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.519282    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\") pod \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") "
Nov 01 03:53:26.519363 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.519318    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\" (UniqueName: \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\") pod \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\" (UID: \"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b\") "
Nov 01 03:53:26.519581 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.519558    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca" pod "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" (UID: "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:26.521403 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.521372    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca" pod "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" (UID: "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:26.521511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.521435    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh" pod "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" (UID: "b74b8a90-9c22-408b-a6db-a5afa3ec2c9b"). InnerVolumeSpecName "kube-api-access-zqvwh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:26.620708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.620677    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:26.620708 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.620705    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:26.620906 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:26.620715    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqvwh\" (UniqueName: \"kubernetes.io/projected/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b-kube-api-access-zqvwh\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:27.250718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:27.250688    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"
Nov 01 03:53:27.250718 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:27.250696    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks" event={"ID":"b74b8a90-9c22-408b-a6db-a5afa3ec2c9b","Type":"ContainerDied","Data":"dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d"}
Nov 01 03:53:27.251055 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:27.250746    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd7849866f0ed70993121e0ae72243ce5b809b75bfc91c59b5839b62ccafc74d"
Nov 01 03:53:27.610028 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:27.609970    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:53:33.520667 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.520628    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Nov 01 03:53:33.735310 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735276    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"]
Nov 01 03:53:33.735647 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735634    4892 cpu_manager.go:414] "containerMap: removing container" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main"
Nov 01 03:53:33.735698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735648    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main"
Nov 01 03:53:33.735698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735662    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:33.735698 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735667    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9af3ab0-f673-44e4-abf0-a80e380650e3" containerName="istio-proxy"
Nov 01 03:53:33.735792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735701    4892 cpu_manager.go:414] "containerMap: removing container" podUID="b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" containerName="revoke-keys"
Nov 01 03:53:33.735792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735707    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" containerName="revoke-keys"
Nov 01 03:53:33.735792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735767    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" containerName="revoke-keys"
Nov 01 03:53:33.735792 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.735778    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e6434e1-0f34-4169-ae1f-5efd29007ce0" containerName="main"
Nov 01 03:53:33.738788 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.738772    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.841783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.841697    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:53:33.841783 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.841714    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:53:33.843759 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.843738    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"]
Nov 01 03:53:33.892941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.892901    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.892941 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.892948    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.893171 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.893123    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wznbt\" (UniqueName: \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.993934 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.993900    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wznbt\" (UniqueName: \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.994115 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.993974    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.994115 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.994021    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.994681 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.994656    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:33.996325 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:33.996300    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:34.055450 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:34.055424    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wznbt\" (UniqueName: \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\") pod \"maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:34.351468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:34.351433    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:34.729524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:34.729444    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"]
Nov 01 03:53:34.733575 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:53:34.733547    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06de2bf1_2ca3_4bb3_a3ec_b092a3b49dc8.slice/crio-e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d WatchSource:0}: Error finding container e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d: Status 404 returned error can't find the container with id e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d
Nov 01 03:53:35.288844 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:35.288802    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s" event={"ID":"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8","Type":"ContainerStarted","Data":"4151caf1749751c3714ca0ff7ebe5bbe0a06ea7eae5228ee1df39c39cacd2d90"}
Nov 01 03:53:35.289041 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:35.288853    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s" event={"ID":"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8","Type":"ContainerStarted","Data":"e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d"}
Nov 01 03:53:35.406713 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:35.406656    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s" podStartSLOduration=2.406642242 podStartE2EDuration="2.406642242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:53:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:53:35.405158126 +0000 UTC m=+2781.821057685" watchObservedRunningTime="2026-11-01 03:53:35.406642242 +0000 UTC m=+2781.822541770"
Nov 01 03:53:36.294827 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:36.294795    4892 generic.go:356] "Generic (PLEG): container finished" podID="06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" containerID="4151caf1749751c3714ca0ff7ebe5bbe0a06ea7eae5228ee1df39c39cacd2d90" exitCode=0
Nov 01 03:53:36.295216 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:36.294840    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s" event={"ID":"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8","Type":"ContainerDied","Data":"4151caf1749751c3714ca0ff7ebe5bbe0a06ea7eae5228ee1df39c39cacd2d90"}
Nov 01 03:53:37.434415 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.434393    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:37.526286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.526254    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\" (UniqueName: \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\") pod \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") "
Nov 01 03:53:37.526455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.526312    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\") pod \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") "
Nov 01 03:53:37.526455 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.526417    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\") pod \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\" (UID: \"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8\") "
Nov 01 03:53:37.526744 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.526714    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca" pod "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" (UID: "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:37.528511 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.528483    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt" pod "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" (UID: "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8"). InnerVolumeSpecName "kube-api-access-wznbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:37.528639 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.528525    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca" pod "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" (UID: "06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:37.627868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.627786    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wznbt\" (UniqueName: \"kubernetes.io/projected/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-kube-api-access-wznbt\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:37.627868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.627817    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:37.627868 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:37.627829    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:38.305152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:38.305117    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"
Nov 01 03:53:38.305152 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:38.305133    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s" event={"ID":"06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8","Type":"ContainerDied","Data":"e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d"}
Nov 01 03:53:38.305367 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:38.305181    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e6150468f80e547bd5eee8803f88329adc7661c31af9c93d452e35bc635c560d"
Nov 01 03:53:40.659191 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.659135    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:53:40.756118 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.756087    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\") pod \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") "
Nov 01 03:53:40.756291 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.756133    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\" (UniqueName: \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\") pod \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\" (UID: \"ee61b8a2-75e4-4482-a70a-f97e1a640b5f\") "
Nov 01 03:53:40.756494 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.756473    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume" pod "ee61b8a2-75e4-4482-a70a-f97e1a640b5f" (UID: "ee61b8a2-75e4-4482-a70a-f97e1a640b5f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:40.758270 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.758244    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5" pod "ee61b8a2-75e4-4482-a70a-f97e1a640b5f" (UID: "ee61b8a2-75e4-4482-a70a-f97e1a640b5f"). InnerVolumeSpecName "kube-api-access-7g6r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:40.857657 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.857623    4892 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-plugins-config-volume\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:40.857657 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:40.857650    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g6r5\" (UniqueName: \"kubernetes.io/projected/ee61b8a2-75e4-4482-a70a-f97e1a640b5f-kube-api-access-7g6r5\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:41.319804 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.319770    4892 generic.go:356] "Generic (PLEG): container finished" podID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerID="74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730" exitCode=0
Nov 01 03:53:41.319997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.319814    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"
Nov 01 03:53:41.319997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.319839    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" event={"ID":"ee61b8a2-75e4-4482-a70a-f97e1a640b5f","Type":"ContainerDied","Data":"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"}
Nov 01 03:53:41.319997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.319885    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs" event={"ID":"ee61b8a2-75e4-4482-a70a-f97e1a640b5f","Type":"ContainerDied","Data":"747a7bdb15d2791b38a32fa5b70fbfdbc91e76081050709aad8de805dcc91e91"}
Nov 01 03:53:41.319997 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.319908    4892 scope.go:122] "RemoveContainer" containerID="74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"
Nov 01 03:53:41.329220 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.329193    4892 scope.go:122] "RemoveContainer" containerID="74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"
Nov 01 03:53:41.329557 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:41.329527    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730\": container with ID starting with 74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730 not found: ID does not exist" containerID="74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"
Nov 01 03:53:41.329610 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.329571    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730"} err="failed to get container status \"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730\": rpc error: code = NotFound desc = could not find container \"74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730\": container with ID starting with 74cdcbc26532f0cd0e51d1cc5b427795fa62c3f7eae940c7abfed1c40666f730 not found: ID does not exist"
Nov 01 03:53:41.495290 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.495255    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:53:41.545628 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.545592    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-c22dd8-5f49746d5f-kt8bs"]
Nov 01 03:53:41.699569 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:41.699474    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" path="/var/lib/kubelet/pods/ee61b8a2-75e4-4482-a70a-f97e1a640b5f/volumes"
Nov 01 03:53:51.098286 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098253    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"]
Nov 01 03:53:51.098680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098597    4892 cpu_manager.go:414] "containerMap: removing container" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing"
Nov 01 03:53:51.098680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098608    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing"
Nov 01 03:53:51.098680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098630    4892 cpu_manager.go:414] "containerMap: removing container" podUID="06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" containerName="revoke-keys"
Nov 01 03:53:51.098680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098637    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" containerName="revoke-keys"
Nov 01 03:53:51.098810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098691    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee61b8a2-75e4-4482-a70a-f97e1a640b5f" containerName="payload-pre-processing"
Nov 01 03:53:51.098810 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.098703    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" containerName="revoke-keys"
Nov 01 03:53:51.103258 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.103237    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.153680 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.153646    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jjp7\" (UniqueName: \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.153847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.153700    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.153847 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.153720    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.159303 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.159274    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"]
Nov 01 03:53:51.207128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.207096    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 03:53:51.207128 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.207108    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 03:53:51.254594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.254561    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.254594 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.254597    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.254815 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.254659    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jjp7\" (UniqueName: \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.255326 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.255300    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.256948 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.256930    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.316387 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.316359    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jjp7\" (UniqueName: \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\") pod \"maas-api-revoke-sub-9895996d-hm4mh\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.416087 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.416006    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:51.787468 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:51.787387    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"]
Nov 01 03:53:51.790077 ip-10-0-134-33 kubenswrapper[4892]: W1101 03:53:51.790048    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee4c3c4_9e96_4e5b_8222_778ce2a36082.slice/crio-b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc WatchSource:0}: Error finding container b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc: Status 404 returned error can't find the container with id b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc
Nov 01 03:53:52.373277 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:52.373240    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh" event={"ID":"aee4c3c4-9e96-4e5b-8222-778ce2a36082","Type":"ContainerStarted","Data":"ca5ab464beffeeb8acacb714f20b3a7a988204d8db17cf1d3491f9c6f22c8bba"}
Nov 01 03:53:52.373665 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:52.373297    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh" event={"ID":"aee4c3c4-9e96-4e5b-8222-778ce2a36082","Type":"ContainerStarted","Data":"b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc"}
Nov 01 03:53:52.493714 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:52.493623    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh" podStartSLOduration=1.493610176 podStartE2EDuration="1.493610176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 03:53:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 03:53:52.491826824 +0000 UTC m=+2798.907726354" watchObservedRunningTime="2026-11-01 03:53:52.493610176 +0000 UTC m=+2798.909509706"
Nov 01 03:53:53.380676 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:53.380642    4892 generic.go:356] "Generic (PLEG): container finished" podID="aee4c3c4-9e96-4e5b-8222-778ce2a36082" containerID="ca5ab464beffeeb8acacb714f20b3a7a988204d8db17cf1d3491f9c6f22c8bba" exitCode=0
Nov 01 03:53:53.381112 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:53.380727    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh" event={"ID":"aee4c3c4-9e96-4e5b-8222-778ce2a36082","Type":"ContainerDied","Data":"ca5ab464beffeeb8acacb714f20b3a7a988204d8db17cf1d3491f9c6f22c8bba"}
Nov 01 03:53:54.515834 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.515806    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:54.586613 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.586584    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\") pod \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") "
Nov 01 03:53:54.586777 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.586635    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\" (UniqueName: \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\") pod \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") "
Nov 01 03:53:54.586832 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.586812    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\") pod \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\" (UID: \"aee4c3c4-9e96-4e5b-8222-778ce2a36082\") "
Nov 01 03:53:54.587212 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.587187    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca" pod "aee4c3c4-9e96-4e5b-8222-778ce2a36082" (UID: "aee4c3c4-9e96-4e5b-8222-778ce2a36082"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 03:53:54.588699 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.588675    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca" pod "aee4c3c4-9e96-4e5b-8222-778ce2a36082" (UID: "aee4c3c4-9e96-4e5b-8222-778ce2a36082"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:53:54.588787 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.588717    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7" pod "aee4c3c4-9e96-4e5b-8222-778ce2a36082" (UID: "aee4c3c4-9e96-4e5b-8222-778ce2a36082"). InnerVolumeSpecName "kube-api-access-2jjp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:53:54.688071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.687998    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:54.688071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.688025    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jjp7\" (UniqueName: \"kubernetes.io/projected/aee4c3c4-9e96-4e5b-8222-778ce2a36082-kube-api-access-2jjp7\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:54.688071 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:54.688036    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/aee4c3c4-9e96-4e5b-8222-778ce2a36082-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:53:55.390313 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:55.390279    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh" event={"ID":"aee4c3c4-9e96-4e5b-8222-778ce2a36082","Type":"ContainerDied","Data":"b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc"}
Nov 01 03:53:55.390495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:55.390334    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1b5a6f30b3d068283408445caee478c5744a7563993075826f6375a6aaff4cc"
Nov 01 03:53:55.390495 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:53:55.390288    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"
Nov 01 03:53:57.609681 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:53:57.609642    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:54:08.618753 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:08.618711    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"]
Nov 01 03:54:08.668262 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:08.668230    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w6-19faa5-nj96s"]
Nov 01 03:54:09.699430 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:09.699396    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8" path="/var/lib/kubelet/pods/06de2bf1-2ca3-4bb3-a3ec-b092a3b49dc8/volumes"
Nov 01 03:54:22.451019 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.450969    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:54:22.451445 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.451189    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager" containerID="cri-o://a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896" gracePeriod=30
Nov 01 03:54:22.712490 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.712425    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:54:22.758145 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.758118    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\" (UniqueName: \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\") pod \"e7934c60-939d-444e-a9cc-d61a30c84adf\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") "
Nov 01 03:54:22.758334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.758184    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\") pod \"e7934c60-939d-444e-a9cc-d61a30c84adf\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") "
Nov 01 03:54:22.758334 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.758312    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\") pod \"e7934c60-939d-444e-a9cc-d61a30c84adf\" (UID: \"e7934c60-939d-444e-a9cc-d61a30c84adf\") "
Nov 01 03:54:22.760349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.760320    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b" pod "e7934c60-939d-444e-a9cc-d61a30c84adf" (UID: "e7934c60-939d-444e-a9cc-d61a30c84adf"). InnerVolumeSpecName "kube-api-access-m874b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 03:54:22.760349 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.760341    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert" pod "e7934c60-939d-444e-a9cc-d61a30c84adf" (UID: "e7934c60-939d-444e-a9cc-d61a30c84adf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:54:22.760489 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.760380    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs" pod "e7934c60-939d-444e-a9cc-d61a30c84adf" (UID: "e7934c60-939d-444e-a9cc-d61a30c84adf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 03:54:22.859462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.859427    4892 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-webhook-cert\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:54:22.859462 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.859457    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7934c60-939d-444e-a9cc-d61a30c84adf-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:54:22.859668 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:22.859472    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m874b\" (UniqueName: \"kubernetes.io/projected/e7934c60-939d-444e-a9cc-d61a30c84adf-kube-api-access-m874b\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 03:54:23.515524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.515492    4892 generic.go:356] "Generic (PLEG): container finished" podID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerID="a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896" exitCode=0
Nov 01 03:54:23.515524 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.515528    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerDied","Data":"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"}
Nov 01 03:54:23.516015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.515539    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8"
Nov 01 03:54:23.516015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.515570    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d96cbcb67-wbhf8" event={"ID":"e7934c60-939d-444e-a9cc-d61a30c84adf","Type":"ContainerDied","Data":"c5716dc64d272d8f046e7f06a4c91cd5090b826dbda31db4e01e9629c6fbf2ee"}
Nov 01 03:54:23.516015 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.515590    4892 scope.go:122] "RemoveContainer" containerID="a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"
Nov 01 03:54:23.531320 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.531298    4892 scope.go:122] "RemoveContainer" containerID="d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"
Nov 01 03:54:23.538759 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.538740    4892 scope.go:122] "RemoveContainer" containerID="a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"
Nov 01 03:54:23.539035 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:54:23.539018    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896\": container with ID starting with a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896 not found: ID does not exist" containerID="a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"
Nov 01 03:54:23.539094 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.539043    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896"} err="failed to get container status \"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896\": rpc error: code = NotFound desc = could not find container \"a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896\": container with ID starting with a9bd54e8e19dd6e40381a98c73ca115c63fccef7d149c8561290b4307271c896 not found: ID does not exist"
Nov 01 03:54:23.539094 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.539060    4892 scope.go:122] "RemoveContainer" containerID="d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"
Nov 01 03:54:23.539323 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:54:23.539305    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6\": container with ID starting with d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6 not found: ID does not exist" containerID="d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"
Nov 01 03:54:23.539366 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.539330    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6"} err="failed to get container status \"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6\": rpc error: code = NotFound desc = could not find container \"d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6\": container with ID starting with d2ea44b20697e36797f3c0902b2b074224816eb035e7354bac6bb42ece0518d6 not found: ID does not exist"
Nov 01 03:54:23.731784 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.731755    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:54:23.781640 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:23.781557    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d96cbcb67-wbhf8"]
Nov 01 03:54:25.703590 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:54:25.703550    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" path="/var/lib/kubelet/pods/e7934c60-939d-444e-a9cc-d61a30c84adf/volumes"
Nov 01 03:54:27.609489 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:54:27.609454    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:54:57.609061 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:54:57.609026    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:55:27.610689 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:55:27.610600    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:55:57.610689 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:55:57.610651    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:56:27.609680 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:56:27.609644    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:56:57.610168 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:56:57.610084    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:57:13.051850 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:13.051813    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"]
Nov 01 03:57:13.101758 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:13.101717    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-32b08a25-ljg4l"]
Nov 01 03:57:13.699702 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:13.699666    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc585305-e654-449e-be9f-a2178b78ed83" path="/var/lib/kubelet/pods/dc585305-e654-449e-be9f-a2178b78ed83/volumes"
Nov 01 03:57:14.068365 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:14.068341    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:57:14.074002 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:14.073969    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 03:57:27.610714 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:57:27.610677    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:57:51.052971 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:51.052932    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"]
Nov 01 03:57:51.104086 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:51.104048    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-73db0618-55ccn"]
Nov 01 03:57:51.700287 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:57:51.700251    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8e18464f-0229-4f09-b792-84711159c93f" path="/var/lib/kubelet/pods/8e18464f-0229-4f09-b792-84711159c93f/volumes"
Nov 01 03:57:57.609167 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:57:57.609123    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:58:14.638859 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:14.638828    4892 scope.go:122] "RemoveContainer" containerID="784dddcea7c66950ed6bae1dcc3b64e7d2f8bc63e54a1031146dee9f8191c285"
Nov 01 03:58:27.051595 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:27.051514    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"]
Nov 01 03:58:27.101407 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:27.101374    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-416a1cd5-xfqks"]
Nov 01 03:58:27.609255 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:58:27.609219    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:58:27.699459 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:27.699421    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b74b8a90-9c22-408b-a6db-a5afa3ec2c9b" path="/var/lib/kubelet/pods/b74b8a90-9c22-408b-a6db-a5afa3ec2c9b/volumes"
Nov 01 03:58:55.049586 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:55.049547    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"]
Nov 01 03:58:55.099064 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:55.099029    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-9895996d-hm4mh"]
Nov 01 03:58:55.699424 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:58:55.699388    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aee4c3c4-9e96-4e5b-8222-778ce2a36082" path="/var/lib/kubelet/pods/aee4c3c4-9e96-4e5b-8222-778ce2a36082/volumes"
Nov 01 03:58:57.609559 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:58:57.609523    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:59:14.660672 ip-10-0-134-33 kubenswrapper[4892]: I1101 03:59:14.660637    4892 scope.go:122] "RemoveContainer" containerID="4f25c7e7b1794bfcf3b15c1416e5a895e87b24101bdcf6ef76f74c3152b1b152"
Nov 01 03:59:27.609353 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:59:27.609314    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 03:59:57.610206 ip-10-0-134-33 kubenswrapper[4892]: E1101 03:59:57.610124    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:00:12.332091 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332056    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"]
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332427    4892 cpu_manager.go:414] "containerMap: removing container" podUID="aee4c3c4-9e96-4e5b-8222-778ce2a36082" containerName="revoke-keys"
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332438    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aee4c3c4-9e96-4e5b-8222-778ce2a36082" containerName="revoke-keys"
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332448    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332453    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332464    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.332516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332469    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.332718 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332526    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="aee4c3c4-9e96-4e5b-8222-778ce2a36082" containerName="revoke-keys"
Nov 01 04:00:12.332718 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332535    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.332718 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.332542    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e7934c60-939d-444e-a9cc-d61a30c84adf" containerName="manager"
Nov 01 04:00:12.335418 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.335402    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.388032 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.387998    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:00:12.438167 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.438135    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:00:12.439491 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.439466    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"]
Nov 01 04:00:12.463229 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.463196    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2659\" (UniqueName: \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.463229 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.463228    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.463448 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.463263    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.564590 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.564554    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2659\" (UniqueName: \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.564590 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.564588    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.564870 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.564624    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.565316 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.565292    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.566934 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.566917    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.626216 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.626150    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2659\" (UniqueName: \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\") pod \"maas-api-revoke-sub-3d4156f1-jgf52\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:12.648139 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:12.648105    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:13.019225 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:13.019193    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"]
Nov 01 04:00:13.022543 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:00:13.022517    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69d1011c_1ee5_4d01_887e_a3c7f1efe4f3.slice/crio-ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5 WatchSource:0}: Error finding container ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5: Status 404 returned error can't find the container with id ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5
Nov 01 04:00:13.202224 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:13.202185    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52" event={"ID":"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3","Type":"ContainerStarted","Data":"ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5"}
Nov 01 04:00:14.209648 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:14.209558    4892 generic.go:356] "Generic (PLEG): container finished" podID="69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" containerID="f3b68bb1566c167d46a6aa49082bdbaeb454ee48bb6baabdfcbcfbb15fdf38c6" exitCode=0
Nov 01 04:00:14.209648 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:14.209611    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52" event={"ID":"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3","Type":"ContainerDied","Data":"f3b68bb1566c167d46a6aa49082bdbaeb454ee48bb6baabdfcbcfbb15fdf38c6"}
Nov 01 04:00:14.680570 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:14.680531    4892 scope.go:122] "RemoveContainer" containerID="ca5ab464beffeeb8acacb714f20b3a7a988204d8db17cf1d3491f9c6f22c8bba"
Nov 01 04:00:14.688503 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:14.688473    4892 scope.go:122] "RemoveContainer" containerID="4151caf1749751c3714ca0ff7ebe5bbe0a06ea7eae5228ee1df39c39cacd2d90"
Nov 01 04:00:14.695667 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:14.695648    4892 scope.go:122] "RemoveContainer" containerID="d5cd4b319c83cb4e913201a4812451f6c93eed5cdfbd929c5f1e7af0237d61a6"
Nov 01 04:00:15.347058 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.347034    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:15.491816 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.491732    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\") pod \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") "
Nov 01 04:00:15.491816 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.491798    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\") pod \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") "
Nov 01 04:00:15.492121 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.491944    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\" (UniqueName: \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\") pod \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\" (UID: \"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3\") "
Nov 01 04:00:15.492193 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.492158    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca" pod "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" (UID: "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:00:15.492334 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.492313    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:00:15.493795 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.493772    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca" pod "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" (UID: "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:00:15.493922 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.493899    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659" pod "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" (UID: "69d1011c-1ee5-4d01-887e-a3c7f1efe4f3"). InnerVolumeSpecName "kube-api-access-v2659". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:00:15.593346 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.593307    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2659\" (UniqueName: \"kubernetes.io/projected/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-kube-api-access-v2659\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:00:15.593346 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:15.593340    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:00:16.218901 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:16.218864    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52" event={"ID":"69d1011c-1ee5-4d01-887e-a3c7f1efe4f3","Type":"ContainerDied","Data":"ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5"}
Nov 01 04:00:16.219090 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:16.218917    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ef4fc4b511e2df04b7cf8c02bfa1133f427d300ffa2827a87bcfbe0fdf82f6d5"
Nov 01 04:00:16.219090 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:00:16.218884    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"
Nov 01 04:00:27.610428 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:00:27.610389    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:00:57.610209 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:00:57.610168    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:01:27.327617 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:01:27.327528    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"]
Nov 01 04:01:27.377458 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:01:27.377421    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-jgf52"]
Nov 01 04:01:27.611690 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:01:27.611608    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:01:27.700043 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:01:27.700008    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" path="/var/lib/kubelet/pods/69d1011c-1ee5-4d01-887e-a3c7f1efe4f3/volumes"
Nov 01 04:01:57.610625 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:01:57.610583    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:02:14.096354 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:14.096325    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:02:14.101335 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:14.101312    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:02:27.609791 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:02:27.609749    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:02:32.723650 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.723611    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:02:32.726125 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.723934    4892 cpu_manager.go:414] "containerMap: removing container" podUID="69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" containerName="revoke-keys"
Nov 01 04:02:32.726125 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.723945    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" containerName="revoke-keys"
Nov 01 04:02:32.726125 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.724046    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="69d1011c-1ee5-4d01-887e-a3c7f1efe4f3" containerName="revoke-keys"
Nov 01 04:02:32.726989 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.726962    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.810261 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.810225    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-feacb8\"/\"kube-root-ca.crt\""
Nov 01 04:02:32.823370 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823335    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2w6w\" (UniqueName: \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.823516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823401    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.823516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823428    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.823516 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823454    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.823640 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823545    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.823683 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.823646    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.830277 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.830243    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-feacb8\"/\"simulator-single-main-feacb8-kserve-self-signed-certs\""
Nov 01 04:02:32.830412 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.830300    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-feacb8\"/\"default-dockercfg-dw5k4\""
Nov 01 04:02:32.830412 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.830324    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-feacb8\"/\"openshift-service-ca.crt\""
Nov 01 04:02:32.831733 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.831714    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:02:32.924705 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924665    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.924914 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924751    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2w6w\" (UniqueName: \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.924914 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924785    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.924914 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924813    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.924914 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924849    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.924914 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.924884    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.925261 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.925231    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.925311 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.925254    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.925311 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.925273    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.927267 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.927242    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.927515 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.927500    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:32.986122 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:32.986057    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2w6w\" (UniqueName: \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\") pod \"simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") " pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:33.039072 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:33.039029    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:33.417608 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:33.417579    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:02:33.420789 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:02:33.420763    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a092c76_53c2_40a8_b90c_bbfdbdc34882.slice/crio-62f7e53f6a5d82eab4b7bcd27acb8e6891255fbf89f992e7648041714ab6234e WatchSource:0}: Error finding container 62f7e53f6a5d82eab4b7bcd27acb8e6891255fbf89f992e7648041714ab6234e: Status 404 returned error can't find the container with id 62f7e53f6a5d82eab4b7bcd27acb8e6891255fbf89f992e7648041714ab6234e
Nov 01 04:02:33.835548 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:33.835515    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" event={"ID":"4a092c76-53c2-40a8-b90c-bbfdbdc34882","Type":"ContainerStarted","Data":"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"}
Nov 01 04:02:33.835548 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:33.835555    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" event={"ID":"4a092c76-53c2-40a8-b90c-bbfdbdc34882","Type":"ContainerStarted","Data":"62f7e53f6a5d82eab4b7bcd27acb8e6891255fbf89f992e7648041714ab6234e"}
Nov 01 04:02:33.953669 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:33.953615    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" podStartSLOduration=1.953601131 podStartE2EDuration="1.953601131s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:02:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:02:33.952150656 +0000 UTC m=+3320.368050186" watchObservedRunningTime="2026-11-01 04:02:33.953601131 +0000 UTC m=+3320.369500660"
Nov 01 04:02:34.039999 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:34.039946    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:43.040198 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:43.040159    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:43.052575 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:43.052547    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:43.893583 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:02:43.893547    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:02:57.610009 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:02:57.609913    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:03:17.774145 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.774110    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"]
Nov 01 04:03:17.777415 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.777399    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:17.830291 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.830242    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:03:17.880238 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.880204    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:03:17.881634 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.881612    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"]
Nov 01 04:03:17.934684 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.934649    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tshz\" (UniqueName: \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:17.934872 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.934691    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:17.934872 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:17.934838    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.035473 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.035381    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.035473 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.035434    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.035661 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.035505    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tshz\" (UniqueName: \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.036110 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.036087    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.037694 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.037676    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.098428 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.098396    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tshz\" (UniqueName: \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\") pod \"maas-api-revoke-sub-b879d329-rtcc9\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.389477 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.389385    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:18.761524 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:18.761492    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"]
Nov 01 04:03:18.765006 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:03:18.764959    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7426123a_93df_48f1_af69_39007b195cd4.slice/crio-d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7 WatchSource:0}: Error finding container d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7: Status 404 returned error can't find the container with id d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7
Nov 01 04:03:19.040234 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:19.040201    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9" event={"ID":"7426123a-93df-48f1-af69-39007b195cd4","Type":"ContainerStarted","Data":"fefc8d502c1c701dcb3cc1c0ccc6ee7ccc90e31d8cb4dd05e2bf8a3e16dc1701"}
Nov 01 04:03:19.040554 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:19.040291    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9" event={"ID":"7426123a-93df-48f1-af69-39007b195cd4","Type":"ContainerStarted","Data":"d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7"}
Nov 01 04:03:19.156401 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:19.156349    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9" podStartSLOduration=2.156335869 podStartE2EDuration="2.156335869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:03:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:03:19.155605554 +0000 UTC m=+3365.571505083" watchObservedRunningTime="2026-11-01 04:03:19.156335869 +0000 UTC m=+3365.572235397"
Nov 01 04:03:20.046887 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:20.046845    4892 generic.go:356] "Generic (PLEG): container finished" podID="7426123a-93df-48f1-af69-39007b195cd4" containerID="fefc8d502c1c701dcb3cc1c0ccc6ee7ccc90e31d8cb4dd05e2bf8a3e16dc1701" exitCode=0
Nov 01 04:03:20.047311 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:20.046924    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9" event={"ID":"7426123a-93df-48f1-af69-39007b195cd4","Type":"ContainerDied","Data":"fefc8d502c1c701dcb3cc1c0ccc6ee7ccc90e31d8cb4dd05e2bf8a3e16dc1701"}
Nov 01 04:03:21.176449 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.176420    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:21.267339 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.267303    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\") pod \"7426123a-93df-48f1-af69-39007b195cd4\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") "
Nov 01 04:03:21.267552 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.267375    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\") pod \"7426123a-93df-48f1-af69-39007b195cd4\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") "
Nov 01 04:03:21.267552 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.267418    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\" (UniqueName: \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\") pod \"7426123a-93df-48f1-af69-39007b195cd4\" (UID: \"7426123a-93df-48f1-af69-39007b195cd4\") "
Nov 01 04:03:21.267816 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.267791    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca" pod "7426123a-93df-48f1-af69-39007b195cd4" (UID: "7426123a-93df-48f1-af69-39007b195cd4"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:03:21.269427 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.269403    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca" pod "7426123a-93df-48f1-af69-39007b195cd4" (UID: "7426123a-93df-48f1-af69-39007b195cd4"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:03:21.269565 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.269543    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz" pod "7426123a-93df-48f1-af69-39007b195cd4" (UID: "7426123a-93df-48f1-af69-39007b195cd4"). InnerVolumeSpecName "kube-api-access-6tshz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:03:21.368580 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.368496    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7426123a-93df-48f1-af69-39007b195cd4-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:21.368580 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.368527    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7426123a-93df-48f1-af69-39007b195cd4-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:21.368580 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:21.368537    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6tshz\" (UniqueName: \"kubernetes.io/projected/7426123a-93df-48f1-af69-39007b195cd4-kube-api-access-6tshz\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:22.056660 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:22.056630    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"
Nov 01 04:03:22.056849 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:22.056667    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9" event={"ID":"7426123a-93df-48f1-af69-39007b195cd4","Type":"ContainerDied","Data":"d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7"}
Nov 01 04:03:22.056849 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:22.056726    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d483111a498df314af955dc746ef2f26c82b753ba7992dff97294e09ef3dedc7"
Nov 01 04:03:27.609957 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:03:27.609922    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:03:34.405851 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:34.405815    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:03:49.424623 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.424560    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" podUID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" containerName="main" containerID="cri-o://3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b" gracePeriod=45
Nov 01 04:03:49.675371 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.675303    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:03:49.721575 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721543    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.721766 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721594    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.721766 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721621    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\" (UniqueName: \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.721766 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721663    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.721766 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721715    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.721766 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721757    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\" (UniqueName: \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\") pod \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\" (UID: \"4a092c76-53c2-40a8-b90c-bbfdbdc34882\") "
Nov 01 04:03:49.722086 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721843    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:03:49.722086 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.721867    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:03:49.722086 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.722042    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:03:49.722196 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.722128    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:49.722196 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.722147    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:49.724014 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.723993    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:03:49.724099 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.724050    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "kube-api-access-j2w6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:03:49.724099 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.724069    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs" pod "4a092c76-53c2-40a8-b90c-bbfdbdc34882" (UID: "4a092c76-53c2-40a8-b90c-bbfdbdc34882"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:03:49.823057 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.823009    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4a092c76-53c2-40a8-b90c-bbfdbdc34882-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:49.823057 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.823042    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:49.823057 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.823050    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4a092c76-53c2-40a8-b90c-bbfdbdc34882-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:49.823057 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:49.823059    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2w6w\" (UniqueName: \"kubernetes.io/projected/4a092c76-53c2-40a8-b90c-bbfdbdc34882-kube-api-access-j2w6w\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:03:50.180039 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.180005    4892 generic.go:356] "Generic (PLEG): container finished" podID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" containerID="3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b" exitCode=0
Nov 01 04:03:50.180204 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.180054    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"
Nov 01 04:03:50.180204 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.180084    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" event={"ID":"4a092c76-53c2-40a8-b90c-bbfdbdc34882","Type":"ContainerDied","Data":"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"}
Nov 01 04:03:50.180204 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.180115    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j" event={"ID":"4a092c76-53c2-40a8-b90c-bbfdbdc34882","Type":"ContainerDied","Data":"62f7e53f6a5d82eab4b7bcd27acb8e6891255fbf89f992e7648041714ab6234e"}
Nov 01 04:03:50.180204 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.180131    4892 scope.go:122] "RemoveContainer" containerID="3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"
Nov 01 04:03:50.191233 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.191136    4892 scope.go:122] "RemoveContainer" containerID="3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"
Nov 01 04:03:50.192122 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:03:50.192093    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b\": container with ID starting with 3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b not found: ID does not exist" containerID="3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"
Nov 01 04:03:50.192216 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.192144    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b"} err="failed to get container status \"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b\": rpc error: code = NotFound desc = could not find container \"3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b\": container with ID starting with 3f1255d8b6be52512c48e70eaed91433f1afce5741b7db13f69bef9acfd8700b not found: ID does not exist"
Nov 01 04:03:50.346664 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.346630    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:03:50.352179 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:50.352153    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-feacb8/simulator-single-main-feacb8-kserve-6f4fc956bf-n9x6j"]
Nov 01 04:03:51.699738 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:51.699704    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" path="/var/lib/kubelet/pods/4a092c76-53c2-40a8-b90c-bbfdbdc34882/volumes"
Nov 01 04:03:57.609726 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:03:57.609689    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:03:58.668381 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668347    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"]
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668672    4892 cpu_manager.go:414] "containerMap: removing container" podUID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" containerName="main"
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668684    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" containerName="main"
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668701    4892 cpu_manager.go:414] "containerMap: removing container" podUID="7426123a-93df-48f1-af69-39007b195cd4" containerName="revoke-keys"
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668706    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7426123a-93df-48f1-af69-39007b195cd4" containerName="revoke-keys"
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668761    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a092c76-53c2-40a8-b90c-bbfdbdc34882" containerName="main"
Nov 01 04:03:58.668770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.668771    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="7426123a-93df-48f1-af69-39007b195cd4" containerName="revoke-keys"
Nov 01 04:03:58.673167 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.673150    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.729069 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.729041    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:03:58.779151 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.779121    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:03:58.780325 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.780307    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"]
Nov 01 04:03:58.801936 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.801915    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.802012 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.801946    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq6fk\" (UniqueName: \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.802012 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.801992    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.903109 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.903073    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.903109 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.903115    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq6fk\" (UniqueName: \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.903387 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.903143    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.903778 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.903756    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.905474 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.905451    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.964556 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.964480    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq6fk\" (UniqueName: \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\") pod \"maas-api-revoke-sub-3d4156f1-q9gqx\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:58.985679 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:58.985650    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:03:59.361380 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:03:59.361341    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"]
Nov 01 04:03:59.365268 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:03:59.365230    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8d39e52_86a5_480f_affe_b5c6a9c1ea58.slice/crio-01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d WatchSource:0}: Error finding container 01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d: Status 404 returned error can't find the container with id 01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d
Nov 01 04:04:00.224432 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:00.224394    4892 generic.go:356] "Generic (PLEG): container finished" podID="f8d39e52-86a5-480f-affe-b5c6a9c1ea58" containerID="6e1a75d3d71387f211d0003f2862746ea06275dc5071aa6c3a8a77cfbb056ad6" exitCode=0
Nov 01 04:04:00.224884 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:00.224446    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx" event={"ID":"f8d39e52-86a5-480f-affe-b5c6a9c1ea58","Type":"ContainerDied","Data":"6e1a75d3d71387f211d0003f2862746ea06275dc5071aa6c3a8a77cfbb056ad6"}
Nov 01 04:04:00.224884 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:00.224493    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx" event={"ID":"f8d39e52-86a5-480f-affe-b5c6a9c1ea58","Type":"ContainerStarted","Data":"01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d"}
Nov 01 04:04:01.356941 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.356915    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:04:01.426306 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.426249    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\") pod \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") "
Nov 01 04:04:01.426306 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.426307    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\" (UniqueName: \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\") pod \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") "
Nov 01 04:04:01.426562 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.426335    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\") pod \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\" (UID: \"f8d39e52-86a5-480f-affe-b5c6a9c1ea58\") "
Nov 01 04:04:01.426790 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.426765    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca" pod "f8d39e52-86a5-480f-affe-b5c6a9c1ea58" (UID: "f8d39e52-86a5-480f-affe-b5c6a9c1ea58"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:04:01.428456 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.428437    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca" pod "f8d39e52-86a5-480f-affe-b5c6a9c1ea58" (UID: "f8d39e52-86a5-480f-affe-b5c6a9c1ea58"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:04:01.428550 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.428484    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk" pod "f8d39e52-86a5-480f-affe-b5c6a9c1ea58" (UID: "f8d39e52-86a5-480f-affe-b5c6a9c1ea58"). InnerVolumeSpecName "kube-api-access-cq6fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:04:01.527158 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.527110    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:04:01.527158 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.527156    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cq6fk\" (UniqueName: \"kubernetes.io/projected/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-kube-api-access-cq6fk\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:04:01.527158 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:01.527167    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f8d39e52-86a5-480f-affe-b5c6a9c1ea58-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:04:02.234770 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:02.234734    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx" event={"ID":"f8d39e52-86a5-480f-affe-b5c6a9c1ea58","Type":"ContainerDied","Data":"01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d"}
Nov 01 04:04:02.234951 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:02.234782    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01ce3d09a135f0a942cbf65cc4d10360436cd6684e220aa174b82bc201b00d0d"
Nov 01 04:04:02.234951 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:04:02.234746    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"
Nov 01 04:04:27.611758 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:04:27.611680    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:04:57.609426 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:04:57.609391    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:05:27.609777 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:05:27.609743    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:05:57.608973 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:05:57.608890    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:06:10.131140 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.131093    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 04:06:10.131796 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.131758    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-ghtkl" podUID="1d0104a2-d41d-4052-9df0-9807d462c73e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 04:06:10.221296 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.221149    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 04:06:10.221296 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.221215    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-wpzmc" podUID="3b8b64b6-a777-4d47-9f48-4f52b5e9008e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 04:06:10.255484 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.255442    4892 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 04:06:10.255694 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:10.255623    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7c5b899d5f-gqkvd" podUID="9c23a0c8-ca80-4ef6-8d51-d2c7977c2762" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 04:06:14.779497 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:06:14.779464    4892 scope.go:122] "RemoveContainer" containerID="f3b68bb1566c167d46a6aa49082bdbaeb454ee48bb6baabdfcbcfbb15fdf38c6"
Nov 01 04:06:27.610593 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:06:27.610554    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:06:57.609281 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:06:57.609242    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:07:04.362640 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.362605    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:07:04.363051 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.362946    4892 cpu_manager.go:414] "containerMap: removing container" podUID="f8d39e52-86a5-480f-affe-b5c6a9c1ea58" containerName="revoke-keys"
Nov 01 04:07:04.363051 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.362958    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8d39e52-86a5-480f-affe-b5c6a9c1ea58" containerName="revoke-keys"
Nov 01 04:07:04.364224 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.363400    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8d39e52-86a5-480f-affe-b5c6a9c1ea58" containerName="revoke-keys"
Nov 01 04:07:04.369572 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.367878    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.442631 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.442595    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:07:04.472962 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.472928    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-077ac6\"/\"kube-root-ca.crt\""
Nov 01 04:07:04.473160 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.472971    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-077ac6\"/\"simulator-single-main-077ac6-kserve-self-signed-certs\""
Nov 01 04:07:04.473160 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.473058    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-077ac6\"/\"openshift-service-ca.crt\""
Nov 01 04:07:04.473160 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.473136    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-077ac6\"/\"default-dockercfg-qt4fr\""
Nov 01 04:07:04.519464 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519422    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfrt9\" (UniqueName: \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.519673 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519489    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.519673 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519598    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.519673 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519659    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.519825 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519750    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.519865 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.519827    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620650 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620565    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfrt9\" (UniqueName: \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620650 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620610    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620650 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620628    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620943 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620660    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620943 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620676    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.620943 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.620702    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.621166 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.621022    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.621225 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.621172    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.621225 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.621210    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.623138 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.623115    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.623315 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.623299    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.685151 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.685111    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfrt9\" (UniqueName: \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\") pod \"simulator-single-main-077ac6-kserve-7c89844f86-zdw6w\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") " pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:04.982401 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:04.982313    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:05.356634 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:05.356592    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:07:05.359234 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:07:05.359206    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod030197fb_54bf_422e_bf17_cef2f0e504c9.slice/crio-8dd60b84adffc51f634baf72cc3c33c1207ff5b0e452d25b07fc33f844a77bf5 WatchSource:0}: Error finding container 8dd60b84adffc51f634baf72cc3c33c1207ff5b0e452d25b07fc33f844a77bf5: Status 404 returned error can't find the container with id 8dd60b84adffc51f634baf72cc3c33c1207ff5b0e452d25b07fc33f844a77bf5
Nov 01 04:07:06.066295 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:06.066254    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" event={"ID":"030197fb-54bf-422e-bf17-cef2f0e504c9","Type":"ContainerStarted","Data":"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"}
Nov 01 04:07:06.066719 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:06.066305    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" event={"ID":"030197fb-54bf-422e-bf17-cef2f0e504c9","Type":"ContainerStarted","Data":"8dd60b84adffc51f634baf72cc3c33c1207ff5b0e452d25b07fc33f844a77bf5"}
Nov 01 04:07:06.185383 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:06.185329    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" podStartSLOduration=2.18531497 podStartE2EDuration="2.18531497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:07:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:07:06.183580167 +0000 UTC m=+3592.599479688" watchObservedRunningTime="2026-11-01 04:07:06.18531497 +0000 UTC m=+3592.601214500"
Nov 01 04:07:06.983153 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:06.983113    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:14.123082 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:14.123055    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:07:14.129723 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:14.129702    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:07:14.983382 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:14.983347    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:14.995971 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:14.995944    4892 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:15.119701 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:15.119673    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:07:27.609343 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:07:27.609306    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:07:49.818200 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.818164    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"]
Nov 01 04:07:49.821962 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.821945    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:49.877660 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.877623    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:07:49.879619 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.879591    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"]
Nov 01 04:07:49.927814 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.927783    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:07:49.936221 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.936197    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:49.936304 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.936242    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:49.936304 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:49.936266    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw4rb\" (UniqueName: \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.037201 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.037168    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.037388 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.037209    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw4rb\" (UniqueName: \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.037388 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.037263    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.037810 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.037792    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.039545 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.039519    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.098674 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.098599    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw4rb\" (UniqueName: \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\") pod \"maas-api-revoke-sub-4d968fb8-znlwk\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.133691 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.133664    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:50.504484 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:50.504454    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"]
Nov 01 04:07:50.507653 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:07:50.507630    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51261805_ed95_4aba_b0de_2286ae55cf4e.slice/crio-ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d WatchSource:0}: Error finding container ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d: Status 404 returned error can't find the container with id ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d
Nov 01 04:07:51.261575 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:51.261545    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk" event={"ID":"51261805-ed95-4aba-b0de-2286ae55cf4e","Type":"ContainerStarted","Data":"eb1704e546aa43767b799834b1ed9b9944411f63a3968427ce6278cb95192ec9"}
Nov 01 04:07:51.261993 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:51.261594    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk" event={"ID":"51261805-ed95-4aba-b0de-2286ae55cf4e","Type":"ContainerStarted","Data":"ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d"}
Nov 01 04:07:51.377890 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:51.377837    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk" podStartSLOduration=2.377823036 podStartE2EDuration="2.377823036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:07:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:07:51.376589929 +0000 UTC m=+3637.792489471" watchObservedRunningTime="2026-11-01 04:07:51.377823036 +0000 UTC m=+3637.793722562"
Nov 01 04:07:52.267268 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:52.267234    4892 generic.go:356] "Generic (PLEG): container finished" podID="51261805-ed95-4aba-b0de-2286ae55cf4e" containerID="eb1704e546aa43767b799834b1ed9b9944411f63a3968427ce6278cb95192ec9" exitCode=0
Nov 01 04:07:52.267657 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:52.267311    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk" event={"ID":"51261805-ed95-4aba-b0de-2286ae55cf4e","Type":"ContainerDied","Data":"eb1704e546aa43767b799834b1ed9b9944411f63a3968427ce6278cb95192ec9"}
Nov 01 04:07:53.400423 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.400395    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:53.470822 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.470793    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\" (UniqueName: \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\") pod \"51261805-ed95-4aba-b0de-2286ae55cf4e\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") "
Nov 01 04:07:53.471005 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.470841    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\") pod \"51261805-ed95-4aba-b0de-2286ae55cf4e\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") "
Nov 01 04:07:53.471005 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.470938    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\") pod \"51261805-ed95-4aba-b0de-2286ae55cf4e\" (UID: \"51261805-ed95-4aba-b0de-2286ae55cf4e\") "
Nov 01 04:07:53.471304 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.471273    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca" pod "51261805-ed95-4aba-b0de-2286ae55cf4e" (UID: "51261805-ed95-4aba-b0de-2286ae55cf4e"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:07:53.472816 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.472793    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb" pod "51261805-ed95-4aba-b0de-2286ae55cf4e" (UID: "51261805-ed95-4aba-b0de-2286ae55cf4e"). InnerVolumeSpecName "kube-api-access-hw4rb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:07:53.472896 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.472878    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca" pod "51261805-ed95-4aba-b0de-2286ae55cf4e" (UID: "51261805-ed95-4aba-b0de-2286ae55cf4e"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:07:53.572357 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.572281    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:07:53.572357 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.572310    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hw4rb\" (UniqueName: \"kubernetes.io/projected/51261805-ed95-4aba-b0de-2286ae55cf4e-kube-api-access-hw4rb\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:07:53.572357 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:53.572320    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/51261805-ed95-4aba-b0de-2286ae55cf4e-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:07:54.280058 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:54.280026    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"
Nov 01 04:07:54.280234 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:54.280027    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk" event={"ID":"51261805-ed95-4aba-b0de-2286ae55cf4e","Type":"ContainerDied","Data":"ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d"}
Nov 01 04:07:54.280234 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:07:54.280153    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae7564583dba5e4821b8df34d7f98edb6b87d0f615a35a3ca0691ee997ca5a4d"
Nov 01 04:07:57.609487 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:07:57.609455    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:08:05.918442 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:05.918405    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:08:20.937447 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:20.937387    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerName="main" containerID="cri-o://68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2" gracePeriod=45
Nov 01 04:08:21.181955 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.181790    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:08:21.212352 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212282    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212352 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212345    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212530 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212396    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212530 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212434    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212530 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212504    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\" (UniqueName: \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212696 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212535    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\" (UniqueName: \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\") pod \"030197fb-54bf-422e-bf17-cef2f0e504c9\" (UID: \"030197fb-54bf-422e-bf17-cef2f0e504c9\") "
Nov 01 04:08:21.212696 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212659    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:08:21.212696 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212678    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:08:21.212859 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212719    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:08:21.212859 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212841    4892 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-model-cache\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.212859 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212853    4892 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-home\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.212969 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.212863    4892 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-tmp-dir\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.214640 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.214608    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 04:08:21.214640 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.214620    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:08:21.214790 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.214674    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9" pod "030197fb-54bf-422e-bf17-cef2f0e504c9" (UID: "030197fb-54bf-422e-bf17-cef2f0e504c9"). InnerVolumeSpecName "kube-api-access-bfrt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:08:21.313688 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.313647    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfrt9\" (UniqueName: \"kubernetes.io/projected/030197fb-54bf-422e-bf17-cef2f0e504c9-kube-api-access-bfrt9\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.313850 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.313693    4892 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/030197fb-54bf-422e-bf17-cef2f0e504c9-tls-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.313850 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.313710    4892 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/030197fb-54bf-422e-bf17-cef2f0e504c9-dshm\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:08:21.403181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.403148    4892 generic.go:356] "Generic (PLEG): container finished" podID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerID="68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2" exitCode=0
Nov 01 04:08:21.403349 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.403192    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"
Nov 01 04:08:21.403349 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.403231    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" event={"ID":"030197fb-54bf-422e-bf17-cef2f0e504c9","Type":"ContainerDied","Data":"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"}
Nov 01 04:08:21.403349 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.403276    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" event={"ID":"030197fb-54bf-422e-bf17-cef2f0e504c9","Type":"ContainerDied","Data":"8dd60b84adffc51f634baf72cc3c33c1207ff5b0e452d25b07fc33f844a77bf5"}
Nov 01 04:08:21.403349 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.403302    4892 scope.go:122] "RemoveContainer" containerID="68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"
Nov 01 04:08:21.413317 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.413295    4892 scope.go:122] "RemoveContainer" containerID="68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"
Nov 01 04:08:21.413610 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:08:21.413591    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2\": container with ID starting with 68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2 not found: ID does not exist" containerID="68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"
Nov 01 04:08:21.413664 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.413620    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2"} err="failed to get container status \"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2\": rpc error: code = NotFound desc = could not find container \"68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2\": container with ID starting with 68e79d9d20654e6e471c5bc0d7ef2ab912749ea4e983f3e46bc54694d3e41ea2 not found: ID does not exist"
Nov 01 04:08:21.569795 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.569759    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:08:21.574455 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.574431    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w"]
Nov 01 04:08:21.700088 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:21.700057    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" path="/var/lib/kubelet/pods/030197fb-54bf-422e-bf17-cef2f0e504c9/volumes"
Nov 01 04:08:22.046362 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:22.046330    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"]
Nov 01 04:08:22.049484 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:22.049455    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-b879d329-rtcc9"]
Nov 01 04:08:22.108998 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:22.108949    4892 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-077ac6/simulator-single-main-077ac6-kserve-7c89844f86-zdw6w" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.138:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 04:08:23.699517 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:08:23.699481    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7426123a-93df-48f1-af69-39007b195cd4" path="/var/lib/kubelet/pods/7426123a-93df-48f1-af69-39007b195cd4/volumes"
Nov 01 04:08:27.610392 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:08:27.610353    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:08:57.610111 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:08:57.610030    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:09:02.047848 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:02.047813    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"]
Nov 01 04:09:02.097762 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:02.097726    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3d4156f1-q9gqx"]
Nov 01 04:09:03.701163 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:03.701128    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8d39e52-86a5-480f-affe-b5c6a9c1ea58" path="/var/lib/kubelet/pods/f8d39e52-86a5-480f-affe-b5c6a9c1ea58/volumes"
Nov 01 04:09:27.609742 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:09:27.609705    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:09:51.690429 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690394    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"]
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690726    4892 cpu_manager.go:414] "containerMap: removing container" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerName="main"
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690740    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerName="main"
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690755    4892 cpu_manager.go:414] "containerMap: removing container" podUID="51261805-ed95-4aba-b0de-2286ae55cf4e" containerName="revoke-keys"
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690761    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51261805-ed95-4aba-b0de-2286ae55cf4e" containerName="revoke-keys"
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690830    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="030197fb-54bf-422e-bf17-cef2f0e504c9" containerName="main"
Nov 01 04:09:51.690921 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.690838    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="51261805-ed95-4aba-b0de-2286ae55cf4e" containerName="revoke-keys"
Nov 01 04:09:51.693734 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.693716    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.746260 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.746220    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:09:51.796442 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.796401    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:09:51.797857 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.797835    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"]
Nov 01 04:09:51.803993 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.803939    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.804110 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.804057    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.804165 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.804110    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8n7v\" (UniqueName: \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.905065 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.905024    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.905065 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.905068    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.905343 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.905099    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8n7v\" (UniqueName: \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.905802 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.905773    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.907443 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.907423    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:51.967060 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:51.966967    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8n7v\" (UniqueName: \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\") pod \"maas-api-revoke-sub-452b6d7b-4znhz\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:52.006152 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:52.006110    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:52.378390 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:52.378363    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"]
Nov 01 04:09:52.381034 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:09:52.381001    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2ab8647_5e46_4e25_b1de_7b472124b44d.slice/crio-ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4 WatchSource:0}: Error finding container ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4: Status 404 returned error can't find the container with id ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4
Nov 01 04:09:52.845697 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:52.845657    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz" event={"ID":"e2ab8647-5e46-4e25-b1de-7b472124b44d","Type":"ContainerStarted","Data":"c0c75e5b29498a3c804b5f79ba0a3a629259d8a6a138fea3c5340431b81be1db"}
Nov 01 04:09:52.846172 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:52.845713    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz" event={"ID":"e2ab8647-5e46-4e25-b1de-7b472124b44d","Type":"ContainerStarted","Data":"ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4"}
Nov 01 04:09:52.962373 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:52.962274    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz" podStartSLOduration=1.962259125 podStartE2EDuration="1.962259125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:09:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:09:52.961388416 +0000 UTC m=+3759.377287947" watchObservedRunningTime="2026-11-01 04:09:52.962259125 +0000 UTC m=+3759.378158653"
Nov 01 04:09:53.853257 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:53.853219    4892 generic.go:356] "Generic (PLEG): container finished" podID="e2ab8647-5e46-4e25-b1de-7b472124b44d" containerID="c0c75e5b29498a3c804b5f79ba0a3a629259d8a6a138fea3c5340431b81be1db" exitCode=0
Nov 01 04:09:53.853792 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:53.853294    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz" event={"ID":"e2ab8647-5e46-4e25-b1de-7b472124b44d","Type":"ContainerDied","Data":"c0c75e5b29498a3c804b5f79ba0a3a629259d8a6a138fea3c5340431b81be1db"}
Nov 01 04:09:55.186270 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.186182    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:55.257167 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.256606    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\") pod \"e2ab8647-5e46-4e25-b1de-7b472124b44d\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") "
Nov 01 04:09:55.257924 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.257428    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\") pod \"e2ab8647-5e46-4e25-b1de-7b472124b44d\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") "
Nov 01 04:09:55.258269 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.257881    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca" pod "e2ab8647-5e46-4e25-b1de-7b472124b44d" (UID: "e2ab8647-5e46-4e25-b1de-7b472124b44d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:09:55.258526 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.258506    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\" (UniqueName: \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\") pod \"e2ab8647-5e46-4e25-b1de-7b472124b44d\" (UID: \"e2ab8647-5e46-4e25-b1de-7b472124b44d\") "
Nov 01 04:09:55.259407 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.259389    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:09:55.280015 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.278234    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v" pod "e2ab8647-5e46-4e25-b1de-7b472124b44d" (UID: "e2ab8647-5e46-4e25-b1de-7b472124b44d"). InnerVolumeSpecName "kube-api-access-w8n7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:09:55.280015 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.279878    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca" pod "e2ab8647-5e46-4e25-b1de-7b472124b44d" (UID: "e2ab8647-5e46-4e25-b1de-7b472124b44d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:09:55.395595 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.395515    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/e2ab8647-5e46-4e25-b1de-7b472124b44d-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:09:55.395595 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.395552    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8n7v\" (UniqueName: \"kubernetes.io/projected/e2ab8647-5e46-4e25-b1de-7b472124b44d-kube-api-access-w8n7v\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:09:55.894361 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.894318    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz" event={"ID":"e2ab8647-5e46-4e25-b1de-7b472124b44d","Type":"ContainerDied","Data":"ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4"}
Nov 01 04:09:55.894578 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.894408    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ecfdc0f21239ad2eae0512274636813adc3ad726957bc3bb4bd9200a04b9b7b4"
Nov 01 04:09:55.894578 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:09:55.894520    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"
Nov 01 04:09:57.610334 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:09:57.610303    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:10:14.835003 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:14.832847    4892 scope.go:122] "RemoveContainer" containerID="fefc8d502c1c701dcb3cc1c0ccc6ee7ccc90e31d8cb4dd05e2bf8a3e16dc1701"
Nov 01 04:10:14.879009 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:14.877221    4892 scope.go:122] "RemoveContainer" containerID="6e1a75d3d71387f211d0003f2862746ea06275dc5071aa6c3a8a77cfbb056ad6"
Nov 01 04:10:19.511091 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.511045    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"]
Nov 01 04:10:19.511617 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.511408    4892 cpu_manager.go:414] "containerMap: removing container" podUID="e2ab8647-5e46-4e25-b1de-7b472124b44d" containerName="revoke-keys"
Nov 01 04:10:19.511617 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.511420    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2ab8647-5e46-4e25-b1de-7b472124b44d" containerName="revoke-keys"
Nov 01 04:10:19.511617 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.511492    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2ab8647-5e46-4e25-b1de-7b472124b44d" containerName="revoke-keys"
Nov 01 04:10:19.514435 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.514413    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.549511 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.549470    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.549780 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.549759    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.550035 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.550014    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbdg8\" (UniqueName: \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.593269 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.593232    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 04:10:19.651868 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.650926    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"]
Nov 01 04:10:19.652245 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.652199    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-h8fq5\""
Nov 01 04:10:19.674030 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.671319    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.676720 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.674660    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbdg8\" (UniqueName: \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.678255 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.678216    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.687990 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.682351    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.712130 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.712098    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.796319 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.796236    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbdg8\" (UniqueName: \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\") pod \"maas-api-revoke-sub-3e31a506-qvh86\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:19.853246 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:19.853199    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:20.304787 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:20.304752    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"]
Nov 01 04:10:20.307832 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:10:20.307806    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod927d2e7e_3762_4a2a_8f9b_92e367437e6c.slice/crio-7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c WatchSource:0}: Error finding container 7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c: Status 404 returned error can't find the container with id 7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c
Nov 01 04:10:21.023081 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:21.022163    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86" event={"ID":"927d2e7e-3762-4a2a-8f9b-92e367437e6c","Type":"ContainerStarted","Data":"2e4dbd1217cfc8669015b27e6462c786740a237a8bf24eb621ff41b45e3566bf"}
Nov 01 04:10:21.023081 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:21.022240    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86" event={"ID":"927d2e7e-3762-4a2a-8f9b-92e367437e6c","Type":"ContainerStarted","Data":"7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c"}
Nov 01 04:10:21.139000 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:21.138922    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86" podStartSLOduration=2.138903539 podStartE2EDuration="2.138903539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:10:21.138033384 +0000 UTC m=+3787.553932916" watchObservedRunningTime="2026-11-01 04:10:21.138903539 +0000 UTC m=+3787.554803069"
Nov 01 04:10:22.027923 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:22.027841    4892 generic.go:356] "Generic (PLEG): container finished" podID="927d2e7e-3762-4a2a-8f9b-92e367437e6c" containerID="2e4dbd1217cfc8669015b27e6462c786740a237a8bf24eb621ff41b45e3566bf" exitCode=0
Nov 01 04:10:22.027923 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:22.027900    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86" event={"ID":"927d2e7e-3762-4a2a-8f9b-92e367437e6c","Type":"ContainerDied","Data":"2e4dbd1217cfc8669015b27e6462c786740a237a8bf24eb621ff41b45e3566bf"}
Nov 01 04:10:23.167654 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.167624    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:23.226176 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.226140    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\" (UniqueName: \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\") pod \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") "
Nov 01 04:10:23.226390 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.226191    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\") pod \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") "
Nov 01 04:10:23.226390 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.226219    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\") pod \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\" (UID: \"927d2e7e-3762-4a2a-8f9b-92e367437e6c\") "
Nov 01 04:10:23.226609 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.226582    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca" pod "927d2e7e-3762-4a2a-8f9b-92e367437e6c" (UID: "927d2e7e-3762-4a2a-8f9b-92e367437e6c"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:10:23.228439 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.228414    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca" pod "927d2e7e-3762-4a2a-8f9b-92e367437e6c" (UID: "927d2e7e-3762-4a2a-8f9b-92e367437e6c"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:10:23.228511 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.228435    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8" pod "927d2e7e-3762-4a2a-8f9b-92e367437e6c" (UID: "927d2e7e-3762-4a2a-8f9b-92e367437e6c"). InnerVolumeSpecName "kube-api-access-kbdg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:10:23.327898 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.327811    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbdg8\" (UniqueName: \"kubernetes.io/projected/927d2e7e-3762-4a2a-8f9b-92e367437e6c-kube-api-access-kbdg8\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:10:23.327898 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.327845    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-openshift-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:10:23.327898 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:23.327857    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/927d2e7e-3762-4a2a-8f9b-92e367437e6c-maas-api-xks-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:10:24.040357 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:24.040327    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"
Nov 01 04:10:24.040658 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:24.040354    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86" event={"ID":"927d2e7e-3762-4a2a-8f9b-92e367437e6c","Type":"ContainerDied","Data":"7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c"}
Nov 01 04:10:24.040797 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:10:24.040686    4892 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7553909279be55d521dc2a29d82be8759f34655e9175f5effbdbab7c628eda6c"
Nov 01 04:10:27.610358 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:10:27.610322    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:10:57.610265 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:10:57.610220    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:11:02.271097 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.271061    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:02.271534 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.271413    4892 cpu_manager.go:414] "containerMap: removing container" podUID="927d2e7e-3762-4a2a-8f9b-92e367437e6c" containerName="revoke-keys"
Nov 01 04:11:02.271534 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.271424    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="927d2e7e-3762-4a2a-8f9b-92e367437e6c" containerName="revoke-keys"
Nov 01 04:11:02.271534 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.271500    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="927d2e7e-3762-4a2a-8f9b-92e367437e6c" containerName="revoke-keys"
Nov 01 04:11:02.274846 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.274828    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.382936 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.382875    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 01 04:11:02.382936 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.382930    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zx9nh\""
Nov 01 04:11:02.383213 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.383052    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 01 04:11:02.384702 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.384672    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:02.425204 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.425159    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgtjw\" (UniqueName: \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.425434 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.425218    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.425434 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.425401    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.425560 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.425460    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.526698 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.526613    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.526698 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.526669    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.526698 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.526699    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgtjw\" (UniqueName: \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.526931 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.526720    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.527453 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.527425    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.529263 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.529235    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.529388 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.529282    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.587925 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.587890    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgtjw\" (UniqueName: \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\") pod \"maas-api-7557fccb47-fc6nn\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") " pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:02.887209 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:02.887116    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:03.264437 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:03.264403    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:03.267584 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:11:03.267557    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36b3afd2_9c18_4610_813c_e5c328ce1793.slice/crio-344b817d6e0fd440033c3932d0219f4f2cbab05b2fc1a7cd15e48e9e7090feed WatchSource:0}: Error finding container 344b817d6e0fd440033c3932d0219f4f2cbab05b2fc1a7cd15e48e9e7090feed: Status 404 returned error can't find the container with id 344b817d6e0fd440033c3932d0219f4f2cbab05b2fc1a7cd15e48e9e7090feed
Nov 01 04:11:03.268870 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:03.268853    4892 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 04:11:04.253961 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:04.253925    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" event={"ID":"36b3afd2-9c18-4610-813c-e5c328ce1793","Type":"ContainerStarted","Data":"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"}
Nov 01 04:11:04.254391 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:04.253997    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" event={"ID":"36b3afd2-9c18-4610-813c-e5c328ce1793","Type":"ContainerStarted","Data":"344b817d6e0fd440033c3932d0219f4f2cbab05b2fc1a7cd15e48e9e7090feed"}
Nov 01 04:11:04.254391 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:04.254020    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:04.372090 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:04.371966    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" podStartSLOduration=1.572853922 podStartE2EDuration="2.371949909s" totalImagesPullingTime="799.095987ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:11:04.371196665 +0000 UTC m=+3830.787096195" watchObservedRunningTime="2026-11-01 04:11:04.371949909 +0000 UTC m=+3830.787849437"
Nov 01 04:11:10.170453 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.170410    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:10.170960 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.170640    4892 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" podUID="36b3afd2-9c18-4610-813c-e5c328ce1793" containerName="maas-api" containerID="cri-o://250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a" gracePeriod=30
Nov 01 04:11:10.176287 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.176262    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:10.220168 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.220133    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"]
Nov 01 04:11:10.223610 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.223587    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.271497 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.271464    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"]
Nov 01 04:11:10.301021 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.300970    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-openshift-service-ca\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.301228 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.301082    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcpx6\" (UniqueName: \"kubernetes.io/projected/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-kube-api-access-gcpx6\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.301228 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.301134    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-metrics-certs\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.301350 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.301313    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-maas-api-tls\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.402419 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.402390    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-maas-api-tls\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.402578 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.402450    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-openshift-service-ca\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.402578 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.402493    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcpx6\" (UniqueName: \"kubernetes.io/projected/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-kube-api-access-gcpx6\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.402578 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.402547    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-metrics-certs\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.403239 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.403214    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-openshift-service-ca\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.405228 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.405202    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-metrics-certs\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.405341 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.405298    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-maas-api-tls\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.423093 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.423020    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:10.463835 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.463800    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcpx6\" (UniqueName: \"kubernetes.io/projected/8aa6eb48-6eaa-460b-9b0b-72a3961a91c7-kube-api-access-gcpx6\") pod \"maas-api-54d5899bf9-dqs8l\" (UID: \"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7\") " pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.503768 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.503732    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\") pod \"36b3afd2-9c18-4610-813c-e5c328ce1793\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") "
Nov 01 04:11:10.504013 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.503810    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\" (UniqueName: \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\") pod \"36b3afd2-9c18-4610-813c-e5c328ce1793\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") "
Nov 01 04:11:10.504013 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.503850    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\") pod \"36b3afd2-9c18-4610-813c-e5c328ce1793\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") "
Nov 01 04:11:10.504013 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.503885    4892 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\") pod \"36b3afd2-9c18-4610-813c-e5c328ce1793\" (UID: \"36b3afd2-9c18-4610-813c-e5c328ce1793\") "
Nov 01 04:11:10.504241 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.504214    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca" pod "36b3afd2-9c18-4610-813c-e5c328ce1793" (UID: "36b3afd2-9c18-4610-813c-e5c328ce1793"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 04:11:10.506080 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.506048    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs" pod "36b3afd2-9c18-4610-813c-e5c328ce1793" (UID: "36b3afd2-9c18-4610-813c-e5c328ce1793"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:11:10.506190 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.506075    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls" pod "36b3afd2-9c18-4610-813c-e5c328ce1793" (UID: "36b3afd2-9c18-4610-813c-e5c328ce1793"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 04:11:10.506190 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.506111    4892 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw" pod "36b3afd2-9c18-4610-813c-e5c328ce1793" (UID: "36b3afd2-9c18-4610-813c-e5c328ce1793"). InnerVolumeSpecName "kube-api-access-zgtjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 04:11:10.560662 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.560623    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:10.605452 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.605411    4892 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgtjw\" (UniqueName: \"kubernetes.io/projected/36b3afd2-9c18-4610-813c-e5c328ce1793-kube-api-access-zgtjw\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:11:10.605452 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.605454    4892 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/36b3afd2-9c18-4610-813c-e5c328ce1793-openshift-service-ca\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:11:10.605664 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.605470    4892 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-metrics-certs\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:11:10.605664 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.605484    4892 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/36b3afd2-9c18-4610-813c-e5c328ce1793-maas-api-tls\") on node \"ip-10-0-134-33.us-west-1.compute.internal\" DevicePath \"\""
Nov 01 04:11:10.936322 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:10.936294    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"]
Nov 01 04:11:10.938964 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:11:10.938936    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8aa6eb48_6eaa_460b_9b0b_72a3961a91c7.slice/crio-3d827ad9ee0a11d24c638936597bb150ed107ffeba207e1d10c60242948ddaa1 WatchSource:0}: Error finding container 3d827ad9ee0a11d24c638936597bb150ed107ffeba207e1d10c60242948ddaa1: Status 404 returned error can't find the container with id 3d827ad9ee0a11d24c638936597bb150ed107ffeba207e1d10c60242948ddaa1
Nov 01 04:11:11.288712 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.288674    4892 generic.go:356] "Generic (PLEG): container finished" podID="36b3afd2-9c18-4610-813c-e5c328ce1793" containerID="250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a" exitCode=0
Nov 01 04:11:11.289238 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.288722    4892 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"
Nov 01 04:11:11.289238 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.288761    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" event={"ID":"36b3afd2-9c18-4610-813c-e5c328ce1793","Type":"ContainerDied","Data":"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"}
Nov 01 04:11:11.289238 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.288816    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn" event={"ID":"36b3afd2-9c18-4610-813c-e5c328ce1793","Type":"ContainerDied","Data":"344b817d6e0fd440033c3932d0219f4f2cbab05b2fc1a7cd15e48e9e7090feed"}
Nov 01 04:11:11.289238 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.288839    4892 scope.go:122] "RemoveContainer" containerID="250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"
Nov 01 04:11:11.290308 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.290137    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l" event={"ID":"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7","Type":"ContainerStarted","Data":"3d827ad9ee0a11d24c638936597bb150ed107ffeba207e1d10c60242948ddaa1"}
Nov 01 04:11:11.298066 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.298049    4892 scope.go:122] "RemoveContainer" containerID="250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"
Nov 01 04:11:11.298362 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:11:11.298342    4892 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a\": container with ID starting with 250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a not found: ID does not exist" containerID="250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"
Nov 01 04:11:11.298428 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.298376    4892 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a"} err="failed to get container status \"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a\": rpc error: code = NotFound desc = could not find container \"250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a\": container with ID starting with 250c27d6859ebe8b2664e009b1ad699993da3feb8d4a9656695d8f4fb946635a not found: ID does not exist"
Nov 01 04:11:11.455299 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.455262    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:11.505316 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.505282    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7557fccb47-fc6nn"]
Nov 01 04:11:11.703297 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:11.703198    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="36b3afd2-9c18-4610-813c-e5c328ce1793" path="/var/lib/kubelet/pods/36b3afd2-9c18-4610-813c-e5c328ce1793/volumes"
Nov 01 04:11:12.297709 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:12.297673    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l" event={"ID":"8aa6eb48-6eaa-460b-9b0b-72a3961a91c7","Type":"ContainerStarted","Data":"d80a15dcd216e6ed42af33f02abf5535af2d4f7bd9b0e1544584dce96e294f36"}
Nov 01 04:11:12.298249 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:12.297801    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:12.421075 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:12.421022    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l" podStartSLOduration=1.554392792 podStartE2EDuration="2.421006897s" totalImagesPullingTime="866.614105ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 04:11:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:11:12.41980591 +0000 UTC m=+3838.835705464" watchObservedRunningTime="2026-11-01 04:11:12.421006897 +0000 UTC m=+3838.836906426"
Nov 01 04:11:18.309864 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:11:18.309832    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54d5899bf9-dqs8l"
Nov 01 04:11:27.610035 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:11:27.609996    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:11:57.611294 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:11:57.611212    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:12:14.151228 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:12:14.151199    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:12:14.159309 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:12:14.159284    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:12:27.611666 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:12:27.611623    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:12:54.047363 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:12:54.047322    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"]
Nov 01 04:12:54.097591 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:12:54.097555    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-4d968fb8-znlwk"]
Nov 01 04:12:55.699952 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:12:55.699918    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51261805-ed95-4aba-b0de-2286ae55cf4e" path="/var/lib/kubelet/pods/51261805-ed95-4aba-b0de-2286ae55cf4e/volumes"
Nov 01 04:12:57.609902 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:12:57.609871    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:13:27.610274 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:13:27.610180    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:13:57.609813 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:13:57.609772    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:14:14.939896 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:14:14.939864    4892 scope.go:122] "RemoveContainer" containerID="eb1704e546aa43767b799834b1ed9b9944411f63a3968427ce6278cb95192ec9"
Nov 01 04:14:27.609876 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:14:27.609840    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:14:55.070020 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:14:55.069924    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"]
Nov 01 04:14:55.119361 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:14:55.119327    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-452b6d7b-4znhz"]
Nov 01 04:14:55.699396 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:14:55.699360    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2ab8647-5e46-4e25-b1de-7b472124b44d" path="/var/lib/kubelet/pods/e2ab8647-5e46-4e25-b1de-7b472124b44d/volumes"
Nov 01 04:14:57.611399 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:14:57.611362    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:15:24.047904 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:24.047863    4892 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"]
Nov 01 04:15:24.097872 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:24.097837    4892 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-3e31a506-qvh86"]
Nov 01 04:15:25.699490 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:25.699453    4892 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="927d2e7e-3762-4a2a-8f9b-92e367437e6c" path="/var/lib/kubelet/pods/927d2e7e-3762-4a2a-8f9b-92e367437e6c/volumes"
Nov 01 04:15:27.609564 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:15:27.609530    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:15:53.860901 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:53.860866    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-p8ncj_b8a0b910-eb4e-4ace-bc8c-cad858b4c819/manager/0.log"
Nov 01 04:15:54.481856 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:54.481822    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-wtt45_f1dd781d-10fb-44e4-b09b-2e6387aa2f5e/manager/0.log"
Nov 01 04:15:55.898203 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:55.898173    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-sq6bx_80c5ac4d-710e-44a9-9394-0de4e216d06c/proxy/0.log"
Nov 01 04:15:56.966319 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:56.966284    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54d5899bf9-dqs8l_8aa6eb48-6eaa-460b-9b0b-72a3961a91c7/maas-api/0.log"
Nov 01 04:15:57.610381 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:15:57.610345    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:15:57.957547 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:15:57.957469    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-cv8c7_92e6cb09-501b-4ae9-96a5-f5b01e520468/postgres/0.log"
Nov 01 04:16:00.146269 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:00.146222    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-c778v_ba648555-2f1d-4cd0-b294-89b1c432c1d2/manager/0.log"
Nov 01 04:16:00.363960 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:00.363926    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-8mskm_9b33f126-741c-4515-842e-a9554a1597d0/kuadrant-console-plugin/0.log"
Nov 01 04:16:02.596349 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:02.596319    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d5d9754f-pzp6s_205d2d1e-a469-47b4-b517-13b220fe8c3f/kube-auth-proxy/0.log"
Nov 01 04:16:03.881379 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:03.881349    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-ghtkl_1d0104a2-d41d-4052-9df0-9807d462c73e/istio-proxy/0.log"
Nov 01 04:16:04.126941 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:04.126908    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-gqkvd_9c23a0c8-ca80-4ef6-8d51-d2c7977c2762/istio-proxy/0.log"
Nov 01 04:16:04.955849 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:04.955819    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-wpzmc_3b8b64b6-a777-4d47-9f48-4f52b5e9008e/istio-proxy/0.log"
Nov 01 04:16:14.965858 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:14.965819    4892 scope.go:122] "RemoveContainer" containerID="c0c75e5b29498a3c804b5f79ba0a3a629259d8a6a138fea3c5340431b81be1db"
Nov 01 04:16:20.256834 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:20.256804    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-p8ncj_b8a0b910-eb4e-4ace-bc8c-cad858b4c819/manager/0.log"
Nov 01 04:16:20.832457 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:20.832421    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-wtt45_f1dd781d-10fb-44e4-b09b-2e6387aa2f5e/manager/0.log"
Nov 01 04:16:22.108391 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:22.108311    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-sq6bx_80c5ac4d-710e-44a9-9394-0de4e216d06c/proxy/0.log"
Nov 01 04:16:22.926275 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:22.926224    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-54d5899bf9-dqs8l_8aa6eb48-6eaa-460b-9b0b-72a3961a91c7/maas-api/0.log"
Nov 01 04:16:23.909614 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:23.909583    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-cv8c7_92e6cb09-501b-4ae9-96a5-f5b01e520468/postgres/0.log"
Nov 01 04:16:25.803886 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:25.803856    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-c778v_ba648555-2f1d-4cd0-b294-89b1c432c1d2/manager/0.log"
Nov 01 04:16:26.041055 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:26.041028    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-8mskm_9b33f126-741c-4515-842e-a9554a1597d0/kuadrant-console-plugin/0.log"
Nov 01 04:16:27.611395 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:16:27.611354    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:16:28.056619 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:28.056588    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d5d9754f-pzp6s_205d2d1e-a469-47b4-b517-13b220fe8c3f/kube-auth-proxy/0.log"
Nov 01 04:16:29.355246 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:29.355215    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-ghtkl_1d0104a2-d41d-4052-9df0-9807d462c73e/istio-proxy/0.log"
Nov 01 04:16:29.581691 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:29.581659    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-gqkvd_9c23a0c8-ca80-4ef6-8d51-d2c7977c2762/istio-proxy/0.log"
Nov 01 04:16:30.446188 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:16:30.446153    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7c5b899d5f-wpzmc_3b8b64b6-a777-4d47-9f48-4f52b5e9008e/istio-proxy/0.log"
Nov 01 04:16:57.611825 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:16:57.611787    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:17:14.179585 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:17:14.179557    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:17:14.188131 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:17:14.188104    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nrvct_69348615-8e95-479b-b360-6991eff92a2e/ovn-acl-logging/0.log"
Nov 01 04:17:14.985958 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:17:14.985925    4892 scope.go:122] "RemoveContainer" containerID="2e4dbd1217cfc8669015b27e6462c786740a237a8bf24eb621ff41b45e3566bf"
Nov 01 04:17:27.609908 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:17:27.609867    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:17:57.608846 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:17:57.608766    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:18:00.935279 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:00.935245    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kkdq4_d4321cab-6240-420f-8920-aa78d1f91105/global-pull-secret-syncer/0.log"
Nov 01 04:18:03.947867 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:03.947833    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-d72l5_43635a20-c47b-45c7-9616-8503d796561e/konnectivity-agent/0.log"
Nov 01 04:18:05.272161 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:05.272133    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-33.us-west-1.compute.internal_726f6b8e587032dd5c6b3f5ea9e72ff8/haproxy/0.log"
Nov 01 04:18:10.703276 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.703239    4892 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"]
Nov 01 04:18:10.703706 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.703622    4892 cpu_manager.go:414] "containerMap: removing container" podUID="36b3afd2-9c18-4610-813c-e5c328ce1793" containerName="maas-api"
Nov 01 04:18:10.703706 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.703635    4892 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="36b3afd2-9c18-4610-813c-e5c328ce1793" containerName="maas-api"
Nov 01 04:18:10.703779 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.703709    4892 memory_manager.go:365] "RemoveStaleState removing state" podUID="36b3afd2-9c18-4610-813c-e5c328ce1793" containerName="maas-api"
Nov 01 04:18:10.706534 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.706515    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.758901 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.758864    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p847s\"/\"openshift-service-ca.crt\""
Nov 01 04:18:10.759083 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.758945    4892 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p847s\"/\"default-dockercfg-d54w4\""
Nov 01 04:18:10.790130 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.790095    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-podres\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.790259 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.790204    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-sys\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.790259 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.790236    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scmtq\" (UniqueName: \"kubernetes.io/projected/ee8d2ee9-c253-4418-a777-8ab53aa997d1-kube-api-access-scmtq\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.790348 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.790303    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-lib-modules\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.790390 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.790375    4892 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-proc\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.809088 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.809056    4892 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p847s\"/\"kube-root-ca.crt\""
Nov 01 04:18:10.810577 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.810558    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"]
Nov 01 04:18:10.891006 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.890939    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-proc\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891026    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-podres\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891081    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-proc\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891098    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-sys\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891139    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scmtq\" (UniqueName: \"kubernetes.io/projected/ee8d2ee9-c253-4418-a777-8ab53aa997d1-kube-api-access-scmtq\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891181 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891152    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-podres\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891351 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891185    4892 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-lib-modules\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891351 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891143    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-sys\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.891351 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.891267    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee8d2ee9-c253-4418-a777-8ab53aa997d1-lib-modules\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:10.951858 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:10.951824    4892 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-scmtq\" (UniqueName: \"kubernetes.io/projected/ee8d2ee9-c253-4418-a777-8ab53aa997d1-kube-api-access-scmtq\") pod \"perf-node-gather-daemonset-ntlhm\" (UID: \"ee8d2ee9-c253-4418-a777-8ab53aa997d1\") " pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:11.019019 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:11.018963    4892 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:11.184491 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:11.184441    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-c778v_ba648555-2f1d-4cd0-b294-89b1c432c1d2/manager/0.log"
Nov 01 04:18:11.394029 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:11.394002    4892 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"]
Nov 01 04:18:11.396841 ip-10-0-134-33 kubenswrapper[4892]: W1101 04:18:11.396810    4892 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podee8d2ee9_c253_4418_a777_8ab53aa997d1.slice/crio-385eb25ba0846ddf12cdd75df5d2f83ba1c208b49c61ed846a330a01b306251d WatchSource:0}: Error finding container 385eb25ba0846ddf12cdd75df5d2f83ba1c208b49c61ed846a330a01b306251d: Status 404 returned error can't find the container with id 385eb25ba0846ddf12cdd75df5d2f83ba1c208b49c61ed846a330a01b306251d
Nov 01 04:18:11.603972 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:11.603943    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-8mskm_9b33f126-741c-4515-842e-a9554a1597d0/kuadrant-console-plugin/0.log"
Nov 01 04:18:12.267502 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:12.267459    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm" event={"ID":"ee8d2ee9-c253-4418-a777-8ab53aa997d1","Type":"ContainerStarted","Data":"2f443f82fd03eeff31aee6984a2645665d4ca0a892ee7551f817bd63c59be1e8"}
Nov 01 04:18:12.267935 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:12.267555    4892 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm" event={"ID":"ee8d2ee9-c253-4418-a777-8ab53aa997d1","Type":"ContainerStarted","Data":"385eb25ba0846ddf12cdd75df5d2f83ba1c208b49c61ed846a330a01b306251d"}
Nov 01 04:18:12.267935 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:12.267596    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:12.384113 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:12.384048    4892 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm" podStartSLOduration=2.38403539 podStartE2EDuration="2.38403539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 04:18:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 04:18:12.383683918 +0000 UTC m=+4258.799583447" watchObservedRunningTime="2026-11-01 04:18:12.38403539 +0000 UTC m=+4258.799934919"
Nov 01 04:18:18.282798 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:18.282766    4892 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p847s/perf-node-gather-daemonset-ntlhm"
Nov 01 04:18:23.646573 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:23.646541    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psfb8_3b8ac128-4c61-4af8-96b7-fd384ae50e07/node-exporter/0.log"
Nov 01 04:18:24.061891 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:24.061858    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psfb8_3b8ac128-4c61-4af8-96b7-fd384ae50e07/kube-rbac-proxy/0.log"
Nov 01 04:18:24.484014 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:24.483904    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-psfb8_3b8ac128-4c61-4af8-96b7-fd384ae50e07/init-textfile/0.log"
Nov 01 04:18:26.175175 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:26.175145    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/prometheus/0.log"
Nov 01 04:18:26.590894 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:26.590866    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/config-reloader/0.log"
Nov 01 04:18:27.006956 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:27.006915    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/thanos-sidecar/0.log"
Nov 01 04:18:27.430331 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:27.430213    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/kube-rbac-proxy-web/0.log"
Nov 01 04:18:27.610787 ip-10-0-134-33 kubenswrapper[4892]: E1101 04:18:27.610751    4892 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-9zmr7" containerName="extractor"
Nov 01 04:18:27.851528 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:27.851495    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/kube-rbac-proxy/0.log"
Nov 01 04:18:28.269811 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:28.269779    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/kube-rbac-proxy-thanos/0.log"
Nov 01 04:18:28.693040 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:28.692898    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_7b78df2f-f10f-457f-acd4-206186a56bf7/init-config-reloader/0.log"
Nov 01 04:18:29.945195 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:29.945157    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-q29mz_c2fda3f7-5f7d-401d-9bc7-42116112b337/prometheus-operator-admission-webhook/0.log"
Nov 01 04:18:34.601652 ip-10-0-134-33 kubenswrapper[4892]: I1101 04:18:34.601620    4892 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d5d9754f-pzp6s_205d2d1e-a469-47b4-b517-13b220fe8c3f/kube-auth-proxy/0.log"